-- Logs begin at Mon 2022-09-05 11:36:02 UTC, end at Mon 2022-09-05 12:53:50 UTC. -- Sep 05 12:08:21.780731 dv-cinder-801354-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 05 12:08:21.789830 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 5 12:08:21 2022] *** Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: nodename: dv-cinder-801354-7-iscsi Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: machine: x86_64 Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: clock source: unix Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: pcre jit disabled Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: detected number of CPU cores: 4 Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: current working directory: / Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: detected binary path: /usr/bin/uwsgi-core Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: your processes number limit is 47680 Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: your memory page size is 4096 bytes Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: detected max file descriptor number: 2048 Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: lock engine: pthread robust mutexes Sep 05 12:08:21.791202 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: thunder lock: enabled Sep 05 12:08:21.792025 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 05 12:08:21.792025 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 05 12:08:21.831003 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: Python main interpreter initialized at 0x55774d6879c0 Sep 05 12:08:21.831003 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: python threads support enabled Sep 05 12:08:21.831003 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: your server socket listen backlog is limited to 100 connections Sep 05 12:08:21.831003 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: your mercy for graceful operations on workers is 90 seconds Sep 05 12:08:21.831356 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: mapped 403077 bytes (393 KB) for 2 cores Sep 05 12:08:21.831470 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: *** Operational MODE: preforking *** Sep 05 12:08:21.831773 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: *** uWSGI is running in multiple interpreter mode *** Sep 05 12:08:21.831773 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: spawned uWSGI master process (pid: 81636) Sep 05 12:08:21.831978 dv-cinder-801354-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 05 12:08:21.833271 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: spawned uWSGI worker 1 (pid: 81648, cores: 1) Sep 05 12:08:21.833271 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: spawned uWSGI worker 2 (pid: 81649, cores: 1) Sep 05 12:08:21.833271 dv-cinder-801354-7-iscsi devstack@keystone.service[81636]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 05 12:08:26.143821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 12:08:26.143821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: from cryptography.utils import int_from_bytes Sep 05 12:08:26.144580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 12:08:26.144580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: from cryptography.utils import int_from_bytes Sep 05 12:08:26.144729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 12:08:26.144729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: from cryptography.utils import int_from_bytes Sep 05 12:08:26.144729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 12:08:26.144729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: from cryptography.utils import int_from_bytes Sep 05 12:08:26.193788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 05 12:08:26.193982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 05 12:08:26.194082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 05 12:08:26.194174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] command line args: [] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 05 12:08:26.194248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 05 12:08:26.194428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] command line args: [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 05 12:08:26.194513 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 05 12:08:26.194599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ================================================================================ {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 05 12:08:26.194697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 05 12:08:26.194836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] admin_token = **** {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.194923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ================================================================================ {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 05 12:08:26.195083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] config_dir = [] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.195165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] admin_token = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.195273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.195452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] config_source = [] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.195609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] config_dir = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.195702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] debug = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.195805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.195894 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.196083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] config_source = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.196154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] debug = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.196219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.196348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.196487 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.196587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.196687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.196793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.196889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.196994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.197099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] list_limit = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.197207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.197309 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] log_config_append = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.197421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.197551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.197654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] log_dir = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.197766 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_config_append = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.197870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] log_file = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.197979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.197979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_dir = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.198201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.198278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_file = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.198406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.198489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.198568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.198682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.198757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.198883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.198961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.198961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.199148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.199148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.199290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.199372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.199451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.199524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.200201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.200362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.200516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.200601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.200717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.200838 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.200921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.201018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.201173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.201257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.201349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.201470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.201599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.201713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.201986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.202066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.202251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] publish_errors = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.202465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.202556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.202636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] publish_errors = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.202729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.202897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.203032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.203113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.203185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.203285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.203385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.203525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.203624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] standard_threads = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.203781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.203867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204010 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] standard_threads = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] use_journal = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] use_json = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] use_stderr = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] use_journal = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] use_syslog = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.204949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] use_json = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.205088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] use_stderr = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.205172 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.205253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] use_syslog = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.205326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.205472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 12:08:26.205557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.205721 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.205879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.205961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.206040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.206213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.206303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.206432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.206520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.206678 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.206763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.206911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.206990 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.207151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.207236 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.207377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.207463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.207644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.207726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.207874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.208225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.208379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.208459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.208612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.208777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.208887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.209007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.209081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.209179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.209310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.209417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.209552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.209668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.209784 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.209917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.210032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.210159 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.210280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.210391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.210519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.210641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.210750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.210896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.210996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.211145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.211235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.211389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.211476 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.211641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.211723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.211881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.211962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.212085 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.212165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.212316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.212395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.212561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.212648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.212804 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.212890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.213032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.213117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.213260 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.213340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.213483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.213567 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.213726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.213814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.213986 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.214065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.214232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.214312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.214463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.214552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.214686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.214765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.214913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.214990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.215139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.215244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.215389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.215473 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.215673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.215756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.219973 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.220097 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.220198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.220295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.220393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.220495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.220607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.220718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.220817 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.220920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.221025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.221126 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.221207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.221333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.221461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.221540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.221672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.221756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.221831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.221947 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222224 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222453 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.222983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223420 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.223949 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224594 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.224967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225123 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.225941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226210 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226664 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.226979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227578 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.227956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228105 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.228993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229224 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.229967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.230112 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.230185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] auth.external = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.230283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.230406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.230491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.230664 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.external = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.230755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.230918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.231001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.231156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] auth.password = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.231235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.231433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] auth.token = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.231515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.231703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.password = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.231783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.231933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] auth.token = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.232014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.232168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.232250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.232393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.232473 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.232632 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.232712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.232859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.232941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.233088 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.233176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.233324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] credential.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.233406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.233548 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.233629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.233785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.233865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.234026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.234105 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.234260 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.234340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.234499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.234578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.234728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.234807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.234962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.235042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.235192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.235272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.235428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.235507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.235696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.235821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.235930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.236103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.236181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.236348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.236508 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.236608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.236689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.236850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.236930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.237100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.237231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.237374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] federation.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.237444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.237607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.237687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.237835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.237925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.238076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.238157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.238298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.238377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.238528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.238614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.238759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.238845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.238986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.239065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.239225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.239313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.239453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.239548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.239721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.239801 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.239952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.240048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.240186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.240265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.240412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.240491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.240654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.240743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.240881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.240960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.241111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.241189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.241341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.241430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.241565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.241645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.241795 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.241874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.242029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.242111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.242271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.242350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.242530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.242610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.242768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.242849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.242995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.243076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.243222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.243302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.243462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.243602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.243729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.243810 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.243975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.244056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.248603 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.248681 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.248681 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.248791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.248791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.248910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.248979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.249931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250004 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250489 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.250948 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.251939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252829 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.252973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.253951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254550 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254685 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254817 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.254961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.255958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.256113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.256202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.256350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.256430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.256573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.256651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.256806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.256894 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.257039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.257120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.257269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.257348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.257499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.257586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.257729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.257809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.257961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.258048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.258201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.258288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.258435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.258516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.258665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.258744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.258898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.259055 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.259132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.259204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.259352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.259434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.259606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.259704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.259840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.259921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.260083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.260170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.260319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.260410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.260549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.260631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.260778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.260857 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.261010 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.261100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] resource.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.261247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.261328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.261486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.261566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.261721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.261809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.261950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.262030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.262760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.262850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.262996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.263086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.263241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.263322 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.263476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.263608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.263739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.263833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.263971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.264050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] role.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.264203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] role.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.264283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] role.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.264427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.264515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] role.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.264663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.264742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.264903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.264982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.265142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.265230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.265381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.265461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.265621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.265710 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.265917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.266018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.266095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.266172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.266332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.266414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.266561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.266643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.266794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.266873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.267033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.267113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.267273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.267360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.267512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.267606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.267787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.267919 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.268153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.268234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.268393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.268472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.268634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.268713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.268875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.268956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.269115 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.269194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.269359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.269442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.269571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.269652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.269820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.269901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.270051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.270131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.270280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.270364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.270504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.270583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.270728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.270807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.270952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.271033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.271198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.271278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.271436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.271512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.271692 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.271776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.271923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.272001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.272147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.272235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.272372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.272452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] token.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.272601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.272680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.272843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.272932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.273076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.273157 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.273305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.273383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.273555 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.273643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.273788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.273869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.274016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.274095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.274251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.274338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.274492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.274572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.274720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.274799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.274955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.275044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.275187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.275275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.275442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.275521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.275709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.275802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.275954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.276043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.276185 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.276274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.276424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.276507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.276681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.connection = **** {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.276760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.276921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.277003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.connection = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.277158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.277237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.277387 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.277466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.277621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.277720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.277874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.277954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.278109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.278189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.278335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.278410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.278559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.278641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.278792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.278872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.279019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.279100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.279252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.279331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.279481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.279588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.279737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.279822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.279974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.280053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.280203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.280281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.280429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.280510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.280658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.280737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.280887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.280966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.281048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81648) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 05 12:08:26.281123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 12:08:26.281266 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81649) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 05 12:08:26.310847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.311016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.311101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.311245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.311332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.311499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.319733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.319904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.351592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.351722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.351833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.351912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.416692 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.417401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.465686 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.465920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.467342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.467603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.632718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.632718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.632718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.632718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.633073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.633176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.633257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.633333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.633411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.633411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.633411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.633411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.654295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.654527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.654682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.654767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.654914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.655042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.655131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.655211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.655351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.655449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.655589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.655679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.655810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.656032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.674806 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.675194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.687311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.687546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.687791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.688016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.688237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.688321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.688454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.688538 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.688670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81648) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.688764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.688925 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.689222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.689453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.689718 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81649) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 12:08:26.712539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.712749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 12:08:26.713085 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.713283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 12:08:26.713543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.713638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.713734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 12:08:26.713814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.714045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.714138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.714256 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 12:08:26.714386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.714522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.714779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.715036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.715329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.724437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.725083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.725083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.725274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.725442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.725722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.725989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.726270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.732250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.733343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.735154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.735434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.735741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.736019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.736268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.736362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.736540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.736645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.736813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.736908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.737083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.737175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.737350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.737443 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.737599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.737689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.737865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.737957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.738133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.738402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.738673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.738935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.760633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.765189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.765295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.765295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.765432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.765432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.767530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.767845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.768127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.768545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.768829 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.769100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.774469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.774751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.775417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.775724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.780999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.781293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.781582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.781867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.782046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.782134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.782363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.782457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.782649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.782732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.782916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.783000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.783193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.783471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.783764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.784059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.798618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.798909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.799734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.800011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.802802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.803893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.806052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.806356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.807115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.807417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.810623 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.811689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.813779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.814898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.817077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.817379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.817664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.817962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.818189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.818276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.818484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.818571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.818765 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.819043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.819333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.819678 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.835169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.835459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.835783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.836012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.836494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.836778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.837044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.837304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.843473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.844837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.845021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.846385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.846569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.846825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.848041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.848316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.850984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.850984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.850984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.850984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.850984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.851276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.851363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.851587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.851871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.852261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.861908 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.862193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.862472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.862749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.863130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.863410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.863566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.863720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.863854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.864140 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.864421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.864802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.865121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.865357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.879726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.881396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.882638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.884384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.885565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.887022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.887799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.889069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.889217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.889354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.889745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.890666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.890956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.891351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.896134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.897588 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.897792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.899236 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.900526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.900922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.901210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.901489 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.902155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.902540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.902835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.903112 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.909401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.909703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.910039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.911176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.911483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.911785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.916934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81648) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.917217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.917499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.917778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.918053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.918363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.918638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.918780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81649) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 12:08:26.918913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.919071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.919200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.919332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.919474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.919635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.919768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.919910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.920040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.920183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.920313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.920441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.920572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81648) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.920702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.920830 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.921076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.921353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.921619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.921880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.922145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81649) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 12:08:26.955467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55774d6879c0 pid: 81648 (default app) Sep 05 12:08:26.957453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55774d6879c0 pid: 81649 (default app) Sep 05 12:08:26.959198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d77a2b-9583-47a7-9919-224f15e7d647 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:26.959482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d77a2b-9583-47a7-9919-224f15e7d647 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:26.959777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d77a2b-9583-47a7-9919-224f15e7d647 None None] PATH_INFO: `/v3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:26.964311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1/1] 10.222.0.62 () {50 vars in 791 bytes} [Mon Sep 5 12:08:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:34.519423 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d472952-6d52-4888-a7f0-1e45cdd23f73 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:34.519977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d472952-6d52-4888-a7f0-1e45cdd23f73 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:34.519977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d472952-6d52-4888-a7f0-1e45cdd23f73 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:34.524241 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1/2] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:08:34.529344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:34.529810 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:34.529908 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:34.599125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: INFO dbcounter [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] Registered counter for database keystone Sep 05 12:08:34.617672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG oslo_db.sqlalchemy.engines [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81648) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 05 12:08:34.624366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writer thread running {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 05 12:08:34.800447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG passlib.handlers.bcrypt [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81648) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 05 12:08:34.800746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG passlib.handlers.bcrypt [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81648) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 05 12:08:34.858695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:34.860346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:34.985996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 2/3] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 05 12:08:35.472173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d472952-6d52-4888-a7f0-1e45cdd23f73 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:35.543382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: INFO dbcounter [None req-7d472952-6d52-4888-a7f0-1e45cdd23f73 None None] Registered counter for database keystone Sep 05 12:08:35.564248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG oslo_db.sqlalchemy.engines [None req-7d472952-6d52-4888-a7f0-1e45cdd23f73 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81649) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 05 12:08:35.570382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writer thread running {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 05 12:08:35.688579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d472952-6d52-4888-a7f0-1e45cdd23f73 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:35.692852 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6ca88-386f-4831-a163-45d919a5b02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:35.693472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6ca88-386f-4831-a163-45d919a5b02b admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:35.693752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6ca88-386f-4831-a163-45d919a5b02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:35.694023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6ca88-386f-4831-a163-45d919a5b02b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:35.702622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6ca88-386f-4831-a163-45d919a5b02b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:35.753753 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b6a6ca88-386f-4831-a163-45d919a5b02b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 05 12:08:35.754039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6ca88-386f-4831-a163-45d919a5b02b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:35.763734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 2/4] 10.222.0.62 () {58 vars in 1324 bytes} [Mon Sep 5 12:08:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:08:35.769611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:35.784745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259eeb9-aa7b-4604-947b-2f8ec9d7ed48 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:35.789205 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a641412-52d5-4cb5-bc44-5f743fa2d912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:35.790296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a641412-52d5-4cb5-bc44-5f743fa2d912 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:35.790580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a641412-52d5-4cb5-bc44-5f743fa2d912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:35.790853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a641412-52d5-4cb5-bc44-5f743fa2d912 admin admin] PATH_INFO: `/v3/projects/d80f282fe9da41e0b08a14ba58840408` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:35.792438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a641412-52d5-4cb5-bc44-5f743fa2d912 admin admin] RBAC: Authorizing `identity:get_project(project_id=d80f282fe9da41e0b08a14ba58840408)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:35.844980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4a641412-52d5-4cb5-bc44-5f743fa2d912 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 05 12:08:35.845321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a641412-52d5-4cb5-bc44-5f743fa2d912 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:35.849876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 3/5] 10.222.0.62 () {56 vars in 1207 bytes} [Mon Sep 5 12:08:35 2022] GET /identity/v3/projects/d80f282fe9da41e0b08a14ba58840408 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:37.702282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151355-aff7-426d-b67e-5667fc5450dd None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:37.702789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151355-aff7-426d-b67e-5667fc5450dd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:37.702873 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151355-aff7-426d-b67e-5667fc5450dd None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:37.706868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 3/6] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:08:37.712446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756109c6-2c60-4d17-9544-f91a57ecf41e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:37.712922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756109c6-2c60-4d17-9544-f91a57ecf41e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:37.713011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756109c6-2c60-4d17-9544-f91a57ecf41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:37.733349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-756109c6-2c60-4d17-9544-f91a57ecf41e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:37.734859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-756109c6-2c60-4d17-9544-f91a57ecf41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:37.742670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 4/7] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 05 12:08:38.006737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a151355-aff7-426d-b67e-5667fc5450dd None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:38.018628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a151355-aff7-426d-b67e-5667fc5450dd None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:38.022748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36904b1f-65a0-4047-9712-2b0e85a07122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:38.023335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36904b1f-65a0-4047-9712-2b0e85a07122 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:38.023641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36904b1f-65a0-4047-9712-2b0e85a07122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:38.023916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36904b1f-65a0-4047-9712-2b0e85a07122 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:38.031370 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36904b1f-65a0-4047-9712-2b0e85a07122 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:38.033143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36904b1f-65a0-4047-9712-2b0e85a07122 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:38.041695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 4/8] 10.222.0.62 () {58 vars in 1324 bytes} [Mon Sep 5 12:08:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:08:38.047312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756109c6-2c60-4d17-9544-f91a57ecf41e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:38.056347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756109c6-2c60-4d17-9544-f91a57ecf41e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:38.061254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb123b-6440-4e6e-8ddf-5f9d298b73c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:38.061878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb123b-6440-4e6e-8ddf-5f9d298b73c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:38.062156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb123b-6440-4e6e-8ddf-5f9d298b73c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:38.062431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb123b-6440-4e6e-8ddf-5f9d298b73c8 admin admin] PATH_INFO: `/v3/users/57929111b78840da8ec77704273d56d2` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:38.063760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb123b-6440-4e6e-8ddf-5f9d298b73c8 admin admin] RBAC: Authorizing `identity:get_user(user_id=57929111b78840da8ec77704273d56d2)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:38.065589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb123b-6440-4e6e-8ddf-5f9d298b73c8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:38.067122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 5/9] 10.222.0.62 () {56 vars in 1198 bytes} [Mon Sep 5 12:08:38 2022] GET /identity/v3/users/57929111b78840da8ec77704273d56d2 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:40.006318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f7924-652b-4a7d-8683-b377731a0e5b None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:40.006835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f7924-652b-4a7d-8683-b377731a0e5b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:40.006952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f7924-652b-4a7d-8683-b377731a0e5b None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:40.008313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 5/10] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:40.013433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f75cb-ef2a-4590-9632-fc85df401704 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:40.013838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f75cb-ef2a-4590-9632-fc85df401704 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:40.013988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f75cb-ef2a-4590-9632-fc85df401704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:40.032323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ab1f75cb-ef2a-4590-9632-fc85df401704 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:40.033505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ab1f75cb-ef2a-4590-9632-fc85df401704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:40.049551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08993b66-59ac-4b9f-afa8-b0ccbd64db68 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:40.049832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08993b66-59ac-4b9f-afa8-b0ccbd64db68 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:40.050567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08993b66-59ac-4b9f-afa8-b0ccbd64db68 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:40.051591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 6/11] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:08:40.056547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a0d6b-da80-4b4c-b764-ae5376c3e59c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:40.056824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a0d6b-da80-4b4c-b764-ae5376c3e59c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:40.057091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a0d6b-da80-4b4c-b764-ae5376c3e59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:40.101262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG passlib.handlers.bcrypt [None req-246a0d6b-da80-4b4c-b764-ae5376c3e59c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81649) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 05 12:08:40.101643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG passlib.handlers.bcrypt [None req-246a0d6b-da80-4b4c-b764-ae5376c3e59c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81649) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 05 12:08:40.117189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 6/12] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:40.150986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-246a0d6b-da80-4b4c-b764-ae5376c3e59c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:40.152610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-246a0d6b-da80-4b4c-b764-ae5376c3e59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:40.253212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 7/13] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:08:40.382748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f75cb-ef2a-4590-9632-fc85df401704 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:40.408037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:40.484493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f75cb-ef2a-4590-9632-fc85df401704 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:40.518850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfe34b-3fb4-444f-b964-5e7b7e6bc394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:40.519237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfe34b-3fb4-444f-b964-5e7b7e6bc394 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:40.519526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfe34b-3fb4-444f-b964-5e7b7e6bc394 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:40.519833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfe34b-3fb4-444f-b964-5e7b7e6bc394 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:40.519919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a0d6b-da80-4b4c-b764-ae5376c3e59c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:40.531408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfe34b-3fb4-444f-b964-5e7b7e6bc394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-04bfe34b-3fb4-444f-b964-5e7b7e6bc394 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:08:40.540320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:40.541369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:08:40.542415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 7/14] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:08:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:08:40.546909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfe34b-3fb4-444f-b964-5e7b7e6bc394 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:40.623244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a0d6b-da80-4b4c-b764-ae5376c3e59c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:40.625534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:40.627556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:40.646033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfe34b-3fb4-444f-b964-5e7b7e6bc394 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:40.660997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a24f7c-a7ee-4f13-b2bc-9f1edac3fcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:40.661853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a24f7c-a7ee-4f13-b2bc-9f1edac3fcef None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:40.662129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a24f7c-a7ee-4f13-b2bc-9f1edac3fcef None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:40.662396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a24f7c-a7ee-4f13-b2bc-9f1edac3fcef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:40.679855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf7694-7dc9-424b-8ff4-bcf19ee0f309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:40.680454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf7694-7dc9-424b-8ff4-bcf19ee0f309 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:40.680913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf7694-7dc9-424b-8ff4-bcf19ee0f309 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:40.681187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf7694-7dc9-424b-8ff4-bcf19ee0f309 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:40.681850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf7694-7dc9-424b-8ff4-bcf19ee0f309 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:40.682152 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf7694-7dc9-424b-8ff4-bcf19ee0f309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:40.683656 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf7694-7dc9-424b-8ff4-bcf19ee0f309 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:40.696086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 8/15] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:08:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:08:40.702157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf7694-7dc9-424b-8ff4-bcf19ee0f309 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:40.730024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a24f7c-a7ee-4f13-b2bc-9f1edac3fcef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:40.732095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a24f7c-a7ee-4f13-b2bc-9f1edac3fcef None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:40.799837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf7694-7dc9-424b-8ff4-bcf19ee0f309 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:40.830558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 8/16] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:08:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:08:40.833196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa274f6-daef-4921-890a-807cca05caa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:40.834085 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa274f6-daef-4921-890a-807cca05caa4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:40.834381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa274f6-daef-4921-890a-807cca05caa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:40.834666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa274f6-daef-4921-890a-807cca05caa4 None admin] PATH_INFO: `/v3/users/57929111b78840da8ec77704273d56d2` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:40.835838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa274f6-daef-4921-890a-807cca05caa4 None admin] RBAC: Authorizing `identity:get_user(user_id=57929111b78840da8ec77704273d56d2)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:40.836396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a24f7c-a7ee-4f13-b2bc-9f1edac3fcef None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:40.837290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa274f6-daef-4921-890a-807cca05caa4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:40.838844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 9/17] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:08:40 2022] GET /identity/v3/users/57929111b78840da8ec77704273d56d2 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:08:40.839215 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:40.844276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa274f6-daef-4921-890a-807cca05caa4 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:40.861002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:40.933853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a24f7c-a7ee-4f13-b2bc-9f1edac3fcef None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:40.943615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa274f6-daef-4921-890a-807cca05caa4 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:40.967424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411855cf-ad45-4afa-b0ac-342e253642dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:40.968299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411855cf-ad45-4afa-b0ac-342e253642dc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:40.968583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411855cf-ad45-4afa-b0ac-342e253642dc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:40.968858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411855cf-ad45-4afa-b0ac-342e253642dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:40.976794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff0f9c-7c13-4ca5-82d7-c3e476342c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:40.977410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff0f9c-7c13-4ca5-82d7-c3e476342c69 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:40.977688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff0f9c-7c13-4ca5-82d7-c3e476342c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:40.977958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff0f9c-7c13-4ca5-82d7-c3e476342c69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:40.979062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff0f9c-7c13-4ca5-82d7-c3e476342c69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:40.980539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff0f9c-7c13-4ca5-82d7-c3e476342c69 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:40.982560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 10/18] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:08:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:08:40.987483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411855cf-ad45-4afa-b0ac-342e253642dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:40.988328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff0f9c-7c13-4ca5-82d7-c3e476342c69 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:40.988840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411855cf-ad45-4afa-b0ac-342e253642dc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-411855cf-ad45-4afa-b0ac-342e253642dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:08:41.007474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:41.008975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:08:41.010095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:08:41.010095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:08:41.010095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:08:41.010095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 9/19] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:08:41.014200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411855cf-ad45-4afa-b0ac-342e253642dc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:41.070884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:41.073851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:41.093129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff0f9c-7c13-4ca5-82d7-c3e476342c69 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:41.116223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411855cf-ad45-4afa-b0ac-342e253642dc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:41.126563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722fe96-f50d-47e1-b471-0a3e571a4dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:41.127174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722fe96-f50d-47e1-b471-0a3e571a4dca None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:41.127458 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722fe96-f50d-47e1-b471-0a3e571a4dca None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:41.127761 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722fe96-f50d-47e1-b471-0a3e571a4dca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:41.128506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722fe96-f50d-47e1-b471-0a3e571a4dca None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.domain.id=default, user.id=57929111b78840da8ec77704273d56d2) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:41.128816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722fe96-f50d-47e1-b471-0a3e571a4dca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:41.130276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722fe96-f50d-47e1-b471-0a3e571a4dca None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:41.137691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 11/20] 10.222.0.62 () {56 vars in 1322 bytes} [Mon Sep 5 12:08:40 2022] GET /identity/v3/role_assignments?user.id=57929111b78840da8ec77704273d56d2&scope.domain.id=default&role.id=705907b9bdec47cebf88b868a6a56356 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:08:41.149906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddaff8-215d-495e-af33-32028e37b5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:41.150533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddaff8-215d-495e-af33-32028e37b5ef None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:41.150808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddaff8-215d-495e-af33-32028e37b5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:41.151076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddaff8-215d-495e-af33-32028e37b5ef None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:41.151773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddaff8-215d-495e-af33-32028e37b5ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:41.152070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddaff8-215d-495e-af33-32028e37b5ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:41.154065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddaff8-215d-495e-af33-32028e37b5ef None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:41.174581 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 10/21] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:08:43.365742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2cc90-c3ba-4c1b-8ced-961b4f91bebc None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:43.366735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2cc90-c3ba-4c1b-8ced-961b4f91bebc None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:43.367219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2cc90-c3ba-4c1b-8ced-961b4f91bebc None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:43.369071 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 12/22] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:43.374084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f08a30-88fb-4e41-9c62-52163c0f75b9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:43.374920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f08a30-88fb-4e41-9c62-52163c0f75b9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:43.375403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f08a30-88fb-4e41-9c62-52163c0f75b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:43.400180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-15f08a30-88fb-4e41-9c62-52163c0f75b9 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:43.401729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-15f08a30-88fb-4e41-9c62-52163c0f75b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:43.492178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 11/23] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:43.503863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517c17e-dff0-418d-8984-8aafccdcff93 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:43.504412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517c17e-dff0-418d-8984-8aafccdcff93 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:43.504891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517c17e-dff0-418d-8984-8aafccdcff93 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:43.506432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 13/24] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:43.512829 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e62e72-e057-402c-ab66-176e7c04d9cb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:43.513349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e62e72-e057-402c-ab66-176e7c04d9cb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:43.513820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e62e72-e057-402c-ab66-176e7c04d9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:43.532797 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f3e62e72-e057-402c-ab66-176e7c04d9cb None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:43.534289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f3e62e72-e057-402c-ab66-176e7c04d9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:43.552182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430e0eb-1452-47b0-bf37-3b97761a6ca8 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:43.552766 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430e0eb-1452-47b0-bf37-3b97761a6ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:43.553258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430e0eb-1452-47b0-bf37-3b97761a6ca8 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:43.554781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 14/25] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:43.560581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410849c4-2f71-4158-8c56-40a740b0ccfc None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:43.561049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410849c4-2f71-4158-8c56-40a740b0ccfc None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:43.561538 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410849c4-2f71-4158-8c56-40a740b0ccfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:43.585161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:43.592483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-410849c4-2f71-4158-8c56-40a740b0ccfc None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:43.598082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-410849c4-2f71-4158-8c56-40a740b0ccfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:43.622486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 12/26] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:43.695752 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 15/27] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:43.874720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e62e72-e057-402c-ab66-176e7c04d9cb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:44.006382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e62e72-e057-402c-ab66-176e7c04d9cb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:44.051607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d49ad-b8b2-4c2e-bd88-cfe0b34fff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:44.052259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d49ad-b8b2-4c2e-bd88-cfe0b34fff1b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:44.052543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d49ad-b8b2-4c2e-bd88-cfe0b34fff1b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:44.052813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d49ad-b8b2-4c2e-bd88-cfe0b34fff1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:44.069926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410849c4-2f71-4158-8c56-40a740b0ccfc None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:44.072372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d49ad-b8b2-4c2e-bd88-cfe0b34fff1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-6b9d49ad-b8b2-4c2e-bd88-cfe0b34fff1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:08:44.079980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:44.080888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:08:44.081662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 13/28] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:08:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:08:44.082925 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd0b8f-262f-4b3a-97a1-15024fbc91bd None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:44.085039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd0b8f-262f-4b3a-97a1-15024fbc91bd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:44.085039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd0b8f-262f-4b3a-97a1-15024fbc91bd None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:44.085318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 14/29] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:44.095574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd0b8f-262f-4b3a-97a1-15024fbc91bd None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:44.167409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:44.187782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:44.213328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410849c4-2f71-4158-8c56-40a740b0ccfc None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:44.242974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd0b8f-262f-4b3a-97a1-15024fbc91bd None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:44.252860 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767fc32-af2b-4214-8bb8-5c743d447942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:44.253673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767fc32-af2b-4214-8bb8-5c743d447942 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:44.254113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767fc32-af2b-4214-8bb8-5c743d447942 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:44.254530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767fc32-af2b-4214-8bb8-5c743d447942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:44.265647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767fc32-af2b-4214-8bb8-5c743d447942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:44.267366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767fc32-af2b-4214-8bb8-5c743d447942 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:44.283282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24077dcc-7f43-4662-8650-505b591c8ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:44.283917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24077dcc-7f43-4662-8650-505b591c8ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:44.284196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24077dcc-7f43-4662-8650-505b591c8ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:44.284468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24077dcc-7f43-4662-8650-505b591c8ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:44.285123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24077dcc-7f43-4662-8650-505b591c8ca9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-5767fc32-af2b-4214-8bb8-5c743d447942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:08:44.285392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:44.286213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:08:44.287056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:08:44.287056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:08:44.287056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:08:44.287211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24077dcc-7f43-4662-8650-505b591c8ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:44.287211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24077dcc-7f43-4662-8650-505b591c8ca9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:44.288636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 16/30] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:08:44.290068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b7073-dfe6-4dad-957b-8706448d94a0 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:44.290580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b7073-dfe6-4dad-957b-8706448d94a0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:44.291083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b7073-dfe6-4dad-957b-8706448d94a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:44.301856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 15/31] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:08:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:44.302802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24077dcc-7f43-4662-8650-505b591c8ca9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:44.317861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1a8b7073-dfe6-4dad-957b-8706448d94a0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:44.319186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1a8b7073-dfe6-4dad-957b-8706448d94a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:44.394396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 17/32] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:44.395602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b7073-dfe6-4dad-957b-8706448d94a0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:44.429746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:44.434773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24077dcc-7f43-4662-8650-505b591c8ca9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:44.472123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9684e9-7272-441c-a87a-a2dfe611addc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:44.472718 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9684e9-7272-441c-a87a-a2dfe611addc None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:44.472992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9684e9-7272-441c-a87a-a2dfe611addc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:44.473261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9684e9-7272-441c-a87a-a2dfe611addc None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:44.473921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9684e9-7272-441c-a87a-a2dfe611addc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:44.475286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9684e9-7272-441c-a87a-a2dfe611addc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:44.475477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:44.533836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b7073-dfe6-4dad-957b-8706448d94a0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:44.570653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5ff19-0d65-427f-bcd9-d5f03c598528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:44.571270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5ff19-0d65-427f-bcd9-d5f03c598528 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:44.571563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5ff19-0d65-427f-bcd9-d5f03c598528 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:44.571903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5ff19-0d65-427f-bcd9-d5f03c598528 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:44.572654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5ff19-0d65-427f-bcd9-d5f03c598528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:44.572950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5ff19-0d65-427f-bcd9-d5f03c598528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:44.574414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5ff19-0d65-427f-bcd9-d5f03c598528 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:44.622871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 18/33] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:44.623801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5ff19-0d65-427f-bcd9-d5f03c598528 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:44.750819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5ff19-0d65-427f-bcd9-d5f03c598528 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:44.772080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:44.786546 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 16/34] 10.222.0.62 () {60 vars in 1131 bytes} [Mon Sep 5 12:08:44 2022] POST /identity/v3/roles => generated 225 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:08:44.787292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc4e2c-81f0-4641-9d39-7a4aff047c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:44.787748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9684e9-7272-441c-a87a-a2dfe611addc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:44.794819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc4e2c-81f0-4641-9d39-7a4aff047c4c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:44.795097 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc4e2c-81f0-4641-9d39-7a4aff047c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:44.795376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc4e2c-81f0-4641-9d39-7a4aff047c4c None admin] PATH_INFO: `/v3/users/57929111b78840da8ec77704273d56d2` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:44.796757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc4e2c-81f0-4641-9d39-7a4aff047c4c None admin] RBAC: Authorizing `identity:get_user(user_id=57929111b78840da8ec77704273d56d2)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:44.798283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc4e2c-81f0-4641-9d39-7a4aff047c4c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:44.800331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 19/35] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:08:44 2022] GET /identity/v3/users/57929111b78840da8ec77704273d56d2 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:44.801086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc4e2c-81f0-4641-9d39-7a4aff047c4c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:44.950188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9684e9-7272-441c-a87a-a2dfe611addc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:44.974668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc4e2c-81f0-4641-9d39-7a4aff047c4c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:45.001913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1bf2b2-feae-4a89-a947-2f325ab4b4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:45.002482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bf2b2-feae-4a89-a947-2f325ab4b4f7 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:45.002762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bf2b2-feae-4a89-a947-2f325ab4b4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:45.003040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bf2b2-feae-4a89-a947-2f325ab4b4f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:45.003989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1bf2b2-feae-4a89-a947-2f325ab4b4f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:45.005574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1bf2b2-feae-4a89-a947-2f325ab4b4f7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:45.026002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f0edf-ee73-4579-820e-6d37ab4c58ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:45.026716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0edf-ee73-4579-820e-6d37ab4c58ae None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:45.027121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0edf-ee73-4579-820e-6d37ab4c58ae None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:45.027524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0edf-ee73-4579-820e-6d37ab4c58ae None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:45.028876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f0edf-ee73-4579-820e-6d37ab4c58ae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:45.030324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f0edf-ee73-4579-820e-6d37ab4c58ae None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:45.142583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 17/36] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:08:44 2022] POST /identity/v3/projects => generated 301 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:08:45.143387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1bf2b2-feae-4a89-a947-2f325ab4b4f7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:45.156767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:45.254637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1bf2b2-feae-4a89-a947-2f325ab4b4f7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:45.276129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 20/37] 10.222.0.62 () {60 vars in 1131 bytes} [Mon Sep 5 12:08:44 2022] POST /identity/v3/roles => generated 219 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:08:45.277575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad36279-3216-4897-964e-77b6e08dc4da None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:45.278428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad36279-3216-4897-964e-77b6e08dc4da None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:45.278708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad36279-3216-4897-964e-77b6e08dc4da None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:45.280192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 21/38] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:45.284844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5f855-c039-4027-9615-d08dcc2cbdcb None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:45.284844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5f855-c039-4027-9615-d08dcc2cbdcb None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:45.284844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5f855-c039-4027-9615-d08dcc2cbdcb None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:45.285963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 22/39] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:45.288267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb7bb-7b31-45ea-897c-390b57b853f4 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:45.289266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb7bb-7b31-45ea-897c-390b57b853f4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:45.289898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb7bb-7b31-45ea-897c-390b57b853f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:45.305068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8e316-d08c-4232-88e4-3012f932bbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:45.305742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8e316-d08c-4232-88e4-3012f932bbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:45.306027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8e316-d08c-4232-88e4-3012f932bbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:45.306307 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8e316-d08c-4232-88e4-3012f932bbf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:45.307487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8e316-d08c-4232-88e4-3012f932bbf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:45.309049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8e316-d08c-4232-88e4-3012f932bbf5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:45.315665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 18/40] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:08:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:45.315855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-90cbb7bb-7b31-45ea-897c-390b57b853f4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:45.315935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38a00-2d32-4847-8603-5b2b6917514f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:45.315935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38a00-2d32-4847-8603-5b2b6917514f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:45.315935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38a00-2d32-4847-8603-5b2b6917514f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:45.316133 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-90cbb7bb-7b31-45ea-897c-390b57b853f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:45.336877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b2d38a00-2d32-4847-8603-5b2b6917514f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:45.338467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b2d38a00-2d32-4847-8603-5b2b6917514f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:45.440896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 19/41] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:45.442567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d38a00-2d32-4847-8603-5b2b6917514f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:45.455473 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:45.494175 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:45.526943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 23/42] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:45.563333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d38a00-2d32-4847-8603-5b2b6917514f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:45.599037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158076c7-4aee-42da-917f-b5c51419f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:45.599732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158076c7-4aee-42da-917f-b5c51419f896 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:45.600019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158076c7-4aee-42da-917f-b5c51419f896 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:45.600299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158076c7-4aee-42da-917f-b5c51419f896 None admin] PATH_INFO: `/v3/domains/default/users/57929111b78840da8ec77704273d56d2/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:45.602461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158076c7-4aee-42da-917f-b5c51419f896 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=57929111b78840da8ec77704273d56d2, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:45.603941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158076c7-4aee-42da-917f-b5c51419f896 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:45.702592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 20/43] 10.222.0.62 () {58 vars in 1361 bytes} [Mon Sep 5 12:08:45 2022] PUT /identity/v3/domains/default/users/57929111b78840da8ec77704273d56d2/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:08:45.713660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbb7bb-7b31-45ea-897c-390b57b853f4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:45.805907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158076c7-4aee-42da-917f-b5c51419f896 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:45.826191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbb7bb-7b31-45ea-897c-390b57b853f4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:45.869098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a819456-bca1-48c3-bf82-12d5f40e2bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:45.870475 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a819456-bca1-48c3-bf82-12d5f40e2bcf None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:45.871092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a819456-bca1-48c3-bf82-12d5f40e2bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:45.872171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a819456-bca1-48c3-bf82-12d5f40e2bcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:45.873480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a819456-bca1-48c3-bf82-12d5f40e2bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:45.875109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a819456-bca1-48c3-bf82-12d5f40e2bcf None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:45.876995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 24/44] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:08:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:45.882134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a819456-bca1-48c3-bf82-12d5f40e2bcf None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:45.883762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:45.918106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158076c7-4aee-42da-917f-b5c51419f896 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:45.936794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:45.954547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da57d1-73ab-4b99-b171-b824248c00cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:45.955167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da57d1-73ab-4b99-b171-b824248c00cb None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:45.955450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da57d1-73ab-4b99-b171-b824248c00cb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:45.955758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da57d1-73ab-4b99-b171-b824248c00cb None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:45.956427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da57d1-73ab-4b99-b171-b824248c00cb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:45.957879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da57d1-73ab-4b99-b171-b824248c00cb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:45.999776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a819456-bca1-48c3-bf82-12d5f40e2bcf None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:46.033073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 21/45] 10.222.0.62 () {60 vars in 1131 bytes} [Mon Sep 5 12:08:45 2022] POST /identity/v3/roles => generated 223 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:08:46.033552 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c162bc-47dd-4285-a07d-9eff60461ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:46.034183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c162bc-47dd-4285-a07d-9eff60461ff4 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:46.034464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c162bc-47dd-4285-a07d-9eff60461ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:46.034739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c162bc-47dd-4285-a07d-9eff60461ff4 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:46.035398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c162bc-47dd-4285-a07d-9eff60461ff4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:46.036928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c162bc-47dd-4285-a07d-9eff60461ff4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:46.138120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 25/46] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:08:45 2022] POST /identity/v3/projects => generated 312 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:08:47.581227 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48284a0-f807-4f47-91f0-65549080c8a2 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:47.581740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48284a0-f807-4f47-91f0-65549080c8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:47.581824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48284a0-f807-4f47-91f0-65549080c8a2 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:47.583123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 22/47] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:08:47.588761 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291e805-4872-40f0-a96f-d3ea040b2b88 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:47.589156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291e805-4872-40f0-a96f-d3ea040b2b88 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:47.589342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291e805-4872-40f0-a96f-d3ea040b2b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:47.607946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7291e805-4872-40f0-a96f-d3ea040b2b88 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:47.609162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7291e805-4872-40f0-a96f-d3ea040b2b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:47.674589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 26/48] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:47.943040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48284a0-f807-4f47-91f0-65549080c8a2 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:48.033246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f85d3-8764-4f50-ba3e-e7d3f6511dbe None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:48.033535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f85d3-8764-4f50-ba3e-e7d3f6511dbe None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:48.033814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f85d3-8764-4f50-ba3e-e7d3f6511dbe None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:48.035146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 27/49] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:08:48.040099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c3dff-dc0c-45a7-9ebc-3493c0a92bc7 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:48.040403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c3dff-dc0c-45a7-9ebc-3493c0a92bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:48.040684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c3dff-dc0c-45a7-9ebc-3493c0a92bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:48.044597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48284a0-f807-4f47-91f0-65549080c8a2 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:48.061236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7b9c3dff-dc0c-45a7-9ebc-3493c0a92bc7 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:48.062427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7b9c3dff-dc0c-45a7-9ebc-3493c0a92bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:48.070740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:48.078441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d03f0-3531-406e-bfd5-947eafefaa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:48.087819 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d03f0-3531-406e-bfd5-947eafefaa5c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:48.087819 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d03f0-3531-406e-bfd5-947eafefaa5c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:48.087819 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d03f0-3531-406e-bfd5-947eafefaa5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:48.088060 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:48.090056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d03f0-3531-406e-bfd5-947eafefaa5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-205d03f0-3531-406e-bfd5-947eafefaa5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:08:48.095413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:48.096565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:08:48.097587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 23/50] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:08:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:08:48.102196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d03f0-3531-406e-bfd5-947eafefaa5c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:48.135406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 28/51] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:08:48.198694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d03f0-3531-406e-bfd5-947eafefaa5c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:48.231526 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fa47f-5a53-4e66-b249-8022799d5e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:48.232148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fa47f-5a53-4e66-b249-8022799d5e49 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:48.232426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fa47f-5a53-4e66-b249-8022799d5e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:48.232700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fa47f-5a53-4e66-b249-8022799d5e49 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:48.233349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fa47f-5a53-4e66-b249-8022799d5e49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:48.233645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fa47f-5a53-4e66-b249-8022799d5e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:48.235026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fa47f-5a53-4e66-b249-8022799d5e49 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:48.244124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 24/52] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:08:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:08:48.249637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c3dff-dc0c-45a7-9ebc-3493c0a92bc7 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:48.349167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c3dff-dc0c-45a7-9ebc-3493c0a92bc7 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:48.382136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fc682-f17d-419f-8f2f-481223193964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:48.382768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fc682-f17d-419f-8f2f-481223193964 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:48.383043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fc682-f17d-419f-8f2f-481223193964 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:48.383317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fc682-f17d-419f-8f2f-481223193964 None admin] PATH_INFO: `/v3/users/57929111b78840da8ec77704273d56d2` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:48.384461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fc682-f17d-419f-8f2f-481223193964 None admin] RBAC: Authorizing `identity:get_user(user_id=57929111b78840da8ec77704273d56d2)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:48.385918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fc682-f17d-419f-8f2f-481223193964 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:48.388012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 29/53] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:08:48 2022] GET /identity/v3/users/57929111b78840da8ec77704273d56d2 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:48.393619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fa47f-5a53-4e66-b249-8022799d5e49 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:48.400951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fc682-f17d-419f-8f2f-481223193964 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:48.425800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:48.508286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fc682-f17d-419f-8f2f-481223193964 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:48.545615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37a194-5835-4f6a-b6c6-d41b13dbe7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:48.546257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37a194-5835-4f6a-b6c6-d41b13dbe7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:48.546542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37a194-5835-4f6a-b6c6-d41b13dbe7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:48.546822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37a194-5835-4f6a-b6c6-d41b13dbe7b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:48.548008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37a194-5835-4f6a-b6c6-d41b13dbe7b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:48.552813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:48.552920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37a194-5835-4f6a-b6c6-d41b13dbe7b6 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:48.552920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 30/54] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:08:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:08:48.556839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37a194-5835-4f6a-b6c6-d41b13dbe7b6 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:48.599554 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fa47f-5a53-4e66-b249-8022799d5e49 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:48.634046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b341e38-ba5a-4681-9bbe-12109c4fed95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:48.634678 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b341e38-ba5a-4681-9bbe-12109c4fed95 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:48.634956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b341e38-ba5a-4681-9bbe-12109c4fed95 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:48.635226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b341e38-ba5a-4681-9bbe-12109c4fed95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:48.636366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b341e38-ba5a-4681-9bbe-12109c4fed95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:48.638405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b341e38-ba5a-4681-9bbe-12109c4fed95 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:48.640009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 25/55] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:08:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:48.645610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b341e38-ba5a-4681-9bbe-12109c4fed95 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:48.648929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:48.658123 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37a194-5835-4f6a-b6c6-d41b13dbe7b6 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:48.691924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab166f39-ef18-4665-91c2-a2570088596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:48.692523 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab166f39-ef18-4665-91c2-a2570088596d None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:48.692803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab166f39-ef18-4665-91c2-a2570088596d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:48.693073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab166f39-ef18-4665-91c2-a2570088596d None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:48.693708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab166f39-ef18-4665-91c2-a2570088596d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:48.695228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab166f39-ef18-4665-91c2-a2570088596d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:48.742904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b341e38-ba5a-4681-9bbe-12109c4fed95 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:48.759897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:48.776675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555690d-a215-4016-8138-1960bf36ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:48.777308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555690d-a215-4016-8138-1960bf36ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:48.777591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555690d-a215-4016-8138-1960bf36ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:48.777886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555690d-a215-4016-8138-1960bf36ccf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:48.778597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555690d-a215-4016-8138-1960bf36ccf2 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.domain.id=default, user.id=57929111b78840da8ec77704273d56d2) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:48.778900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555690d-a215-4016-8138-1960bf36ccf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:48.780341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555690d-a215-4016-8138-1960bf36ccf2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:48.788367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 26/56] 10.222.0.62 () {56 vars in 1322 bytes} [Mon Sep 5 12:08:48 2022] GET /identity/v3/role_assignments?user.id=57929111b78840da8ec77704273d56d2&scope.domain.id=default&role.id=705907b9bdec47cebf88b868a6a56356 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:08:48.818059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 31/57] 10.222.0.62 () {60 vars in 1140 bytes} [Mon Sep 5 12:08:48 2022] POST /identity/v3/projects => generated 298 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:08:50.744996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84520db2-359d-4ddb-97cc-67e480677954 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:50.745509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84520db2-359d-4ddb-97cc-67e480677954 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:50.745621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84520db2-359d-4ddb-97cc-67e480677954 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:50.747136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 27/58] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:50.752208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e70706-9ffe-4c7e-a9db-4fd09020a8e1 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:50.752617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e70706-9ffe-4c7e-a9db-4fd09020a8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:50.752756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e70706-9ffe-4c7e-a9db-4fd09020a8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:50.771031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-08e70706-9ffe-4c7e-a9db-4fd09020a8e1 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:50.772252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-08e70706-9ffe-4c7e-a9db-4fd09020a8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:50.836550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 32/59] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:51.110329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84520db2-359d-4ddb-97cc-67e480677954 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:51.211299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84520db2-359d-4ddb-97cc-67e480677954 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:51.243500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20351a-be31-4e00-8041-46fa77a8bfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:51.244146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20351a-be31-4e00-8041-46fa77a8bfce None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:51.244429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20351a-be31-4e00-8041-46fa77a8bfce None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:51.244705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20351a-be31-4e00-8041-46fa77a8bfce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:51.245801 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20351a-be31-4e00-8041-46fa77a8bfce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:51.247855 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20351a-be31-4e00-8041-46fa77a8bfce None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:51.249396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 28/60] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:08:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:08:51.255091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e70706-9ffe-4c7e-a9db-4fd09020a8e1 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:51.323369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:51.354024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e70706-9ffe-4c7e-a9db-4fd09020a8e1 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:51.387222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528ac4b-f843-4561-bd56-00bb9cbda235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:51.387890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528ac4b-f843-4561-bd56-00bb9cbda235 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:51.388172 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528ac4b-f843-4561-bd56-00bb9cbda235 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:51.388445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528ac4b-f843-4561-bd56-00bb9cbda235 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:51.389082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528ac4b-f843-4561-bd56-00bb9cbda235 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:51.390525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528ac4b-f843-4561-bd56-00bb9cbda235 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:51.540645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 33/61] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:08:51 2022] POST /identity/v3/users => generated 275 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:08:54.510771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16e2e-441c-4669-9807-d78b36ffcfe4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:54.511270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16e2e-441c-4669-9807-d78b36ffcfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:54.511270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16e2e-441c-4669-9807-d78b36ffcfe4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:54.512912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 29/62] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:54.518250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c7d0d-f35b-4775-aded-fbc12b146672 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:54.518665 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c7d0d-f35b-4775-aded-fbc12b146672 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:54.518823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c7d0d-f35b-4775-aded-fbc12b146672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:54.544882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-364c7d0d-f35b-4775-aded-fbc12b146672 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:54.546138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-364c7d0d-f35b-4775-aded-fbc12b146672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:54.577792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c73a2-3984-4f8a-ac2f-6f2e75d6efee None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:54.578392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c73a2-3984-4f8a-ac2f-6f2e75d6efee None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:54.578902 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c73a2-3984-4f8a-ac2f-6f2e75d6efee None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:54.580728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 30/63] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:54.609679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bebfa9-0d8d-4a18-b4f9-c9cb3d9d6d20 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:54.610266 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bebfa9-0d8d-4a18-b4f9-c9cb3d9d6d20 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:54.610740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bebfa9-0d8d-4a18-b4f9-c9cb3d9d6d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:54.626618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 34/64] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:54.652528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-20bebfa9-0d8d-4a18-b4f9-c9cb3d9d6d20 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:54.654614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-20bebfa9-0d8d-4a18-b4f9-c9cb3d9d6d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:54.727705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:54.757643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 31/65] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:08:54.939764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30379db-7672-435a-881e-a67f5f673a78 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:54.940371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30379db-7672-435a-881e-a67f5f673a78 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:54.940824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30379db-7672-435a-881e-a67f5f673a78 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:54.942614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 35/66] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:54.947728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559f90-2516-4324-a7b0-12e5d8c25886 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:54.947950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644f021-ba9b-48aa-89df-b4a49e6fd527 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:54.948247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644f021-ba9b-48aa-89df-b4a49e6fd527 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:54.948527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644f021-ba9b-48aa-89df-b4a49e6fd527 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:54.948876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559f90-2516-4324-a7b0-12e5d8c25886 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:54.949353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559f90-2516-4324-a7b0-12e5d8c25886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:54.949907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 32/67] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:54.955010 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d425b38-0b64-464f-b18f-2d0af8146c45 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:54.955296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d425b38-0b64-464f-b18f-2d0af8146c45 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:54.955600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d425b38-0b64-464f-b18f-2d0af8146c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:54.971182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-da559f90-2516-4324-a7b0-12e5d8c25886 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:54.972415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-da559f90-2516-4324-a7b0-12e5d8c25886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:54.984046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6d425b38-0b64-464f-b18f-2d0af8146c45 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:54.985963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6d425b38-0b64-464f-b18f-2d0af8146c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:55.044421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:55.066464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 33/68] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:08:55.075911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d425b38-0b64-464f-b18f-2d0af8146c45 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:55.077758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 36/69] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:08:55.100193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559f90-2516-4324-a7b0-12e5d8c25886 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:55.241284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d425b38-0b64-464f-b18f-2d0af8146c45 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:55.254514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:55.282037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455632a-130a-4519-9e15-1ee9f246974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:55.282972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455632a-130a-4519-9e15-1ee9f246974a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.283390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455632a-130a-4519-9e15-1ee9f246974a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.284030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455632a-130a-4519-9e15-1ee9f246974a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.296173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455632a-130a-4519-9e15-1ee9f246974a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-b455632a-130a-4519-9e15-1ee9f246974a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:08:55.301806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:55.302846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:08:55.303834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:08:55.306327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 34/70] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:08:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:08:55.312198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455632a-130a-4519-9e15-1ee9f246974a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:55.324108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559f90-2516-4324-a7b0-12e5d8c25886 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:55.369522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f50dbb-42cb-4f35-a3e1-a233e6f02a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:55.370170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f50dbb-42cb-4f35-a3e1-a233e6f02a10 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.370453 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f50dbb-42cb-4f35-a3e1-a233e6f02a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.370729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f50dbb-42cb-4f35-a3e1-a233e6f02a10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.383416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f50dbb-42cb-4f35-a3e1-a233e6f02a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-95f50dbb-42cb-4f35-a3e1-a233e6f02a10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:08:55.389041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:55.390073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:08:55.390869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 37/71] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:08:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:08:55.391495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f50dbb-42cb-4f35-a3e1-a233e6f02a10 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:55.440723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:55.450503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455632a-130a-4519-9e15-1ee9f246974a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:55.483630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafd506-c204-4d1f-af8a-12f1601e88e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:55.484243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafd506-c204-4d1f-af8a-12f1601e88e4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.484528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafd506-c204-4d1f-af8a-12f1601e88e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.484804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafd506-c204-4d1f-af8a-12f1601e88e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.485499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafd506-c204-4d1f-af8a-12f1601e88e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:55.485805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafd506-c204-4d1f-af8a-12f1601e88e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:55.487178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafd506-c204-4d1f-af8a-12f1601e88e4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:55.491280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f50dbb-42cb-4f35-a3e1-a233e6f02a10 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:55.495428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 35/72] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:08:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:08:55.496000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafd506-c204-4d1f-af8a-12f1601e88e4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:55.526954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b2491-7e85-4a6c-a206-64a58545e48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:55.527607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b2491-7e85-4a6c-a206-64a58545e48c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.527889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b2491-7e85-4a6c-a206-64a58545e48c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.528163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b2491-7e85-4a6c-a206-64a58545e48c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.528962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:55.538129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b2491-7e85-4a6c-a206-64a58545e48c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-ae0b2491-7e85-4a6c-a206-64a58545e48c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:08:55.543082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:55.552535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 38/73] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:08:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:08:55.553343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bd1b3-353c-4656-91c3-3733cfb34732 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.554051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bd1b3-353c-4656-91c3-3733cfb34732 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.554051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bd1b3-353c-4656-91c3-3733cfb34732 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.554051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 39/74] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:55.554051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bd1b3-353c-4656-91c3-3733cfb34732 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:55.597088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafd506-c204-4d1f-af8a-12f1601e88e4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:55.630168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db55f8-d34a-440b-a863-05b4d2a5836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:55.630854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db55f8-d34a-440b-a863-05b4d2a5836e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.631141 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db55f8-d34a-440b-a863-05b4d2a5836e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.631413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db55f8-d34a-440b-a863-05b4d2a5836e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.642076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db55f8-d34a-440b-a863-05b4d2a5836e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-92db55f8-d34a-440b-a863-05b4d2a5836e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:08:55.647242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:08:55.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 36/75] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:08:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:08:55.649450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db55f8-d34a-440b-a863-05b4d2a5836e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:55.655715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bd1b3-353c-4656-91c3-3733cfb34732 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:55.665146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:55.689835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a71ccf-d9be-4d79-9b33-bc2eda5dd9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:55.690450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71ccf-d9be-4d79-9b33-bc2eda5dd9ff None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.690734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71ccf-d9be-4d79-9b33-bc2eda5dd9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.691014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71ccf-d9be-4d79-9b33-bc2eda5dd9ff None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.691768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a71ccf-d9be-4d79-9b33-bc2eda5dd9ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:55.692083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a71ccf-d9be-4d79-9b33-bc2eda5dd9ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:55.693486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a71ccf-d9be-4d79-9b33-bc2eda5dd9ff None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:55.702580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 40/76] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:08:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:55.704026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a71ccf-d9be-4d79-9b33-bc2eda5dd9ff None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:55.744369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:55.753294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db55f8-d34a-440b-a863-05b4d2a5836e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:55.785778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1f8e2-ab10-4639-9f5a-c3a8191ef94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:55.786404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1f8e2-ab10-4639-9f5a-c3a8191ef94d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.786681 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1f8e2-ab10-4639-9f5a-c3a8191ef94d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.786950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1f8e2-ab10-4639-9f5a-c3a8191ef94d None admin] PATH_INFO: `/v3/users/6b6757539b0548b8a38441652f18bce9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.801273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a71ccf-d9be-4d79-9b33-bc2eda5dd9ff None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:55.801454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1f8e2-ab10-4639-9f5a-c3a8191ef94d None admin] RBAC: Authorizing `identity:get_user(user_id=6b6757539b0548b8a38441652f18bce9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:55.802909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1f8e2-ab10-4639-9f5a-c3a8191ef94d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:55.804547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 37/77] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:08:55 2022] GET /identity/v3/users/6b6757539b0548b8a38441652f18bce9 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:55.805668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b5db6-13f3-43db-b492-b6a7ecd8a784 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.805948 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b5db6-13f3-43db-b492-b6a7ecd8a784 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.806221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b5db6-13f3-43db-b492-b6a7ecd8a784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.823877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-596b5db6-13f3-43db-b492-b6a7ecd8a784 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:55.825056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-596b5db6-13f3-43db-b492-b6a7ecd8a784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:08:55.835733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968a342-171f-484c-a8c5-5b33a83edf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:55.836346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968a342-171f-484c-a8c5-5b33a83edf63 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.836630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968a342-171f-484c-a8c5-5b33a83edf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.836903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968a342-171f-484c-a8c5-5b33a83edf63 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.837556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968a342-171f-484c-a8c5-5b33a83edf63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:55.837861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968a342-171f-484c-a8c5-5b33a83edf63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:55.839247 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968a342-171f-484c-a8c5-5b33a83edf63 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:55.847303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 41/78] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:08:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:55.847905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968a342-171f-484c-a8c5-5b33a83edf63 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:55.879377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:55.895888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 38/79] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:08:55.896750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b5db6-13f3-43db-b492-b6a7ecd8a784 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:55.952742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968a342-171f-484c-a8c5-5b33a83edf63 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:55.982716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:55.988542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52aa61-9da8-4e33-8557-3f40bbd10595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:55.989173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52aa61-9da8-4e33-8557-3f40bbd10595 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:55.989444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52aa61-9da8-4e33-8557-3f40bbd10595 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:55.989723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52aa61-9da8-4e33-8557-3f40bbd10595 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:55.990388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52aa61-9da8-4e33-8557-3f40bbd10595 None admin] RBAC: Adding query filter params (name=member) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:55.990691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52aa61-9da8-4e33-8557-3f40bbd10595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:55.992141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52aa61-9da8-4e33-8557-3f40bbd10595 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:56.002610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 42/80] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:08:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:56.002762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b5db6-13f3-43db-b492-b6a7ecd8a784 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:56.007727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52aa61-9da8-4e33-8557-3f40bbd10595 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:56.039960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaaaf97-eec6-4dba-be34-419996b5c59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:56.040599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaaf97-eec6-4dba-be34-419996b5c59d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:56.040878 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaaf97-eec6-4dba-be34-419996b5c59d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:56.041151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaaf97-eec6-4dba-be34-419996b5c59d None admin] PATH_INFO: `/v3/users/57929111b78840da8ec77704273d56d2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:56.042311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaaaf97-eec6-4dba-be34-419996b5c59d None admin] RBAC: Authorizing `identity:get_user(user_id=57929111b78840da8ec77704273d56d2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:56.044336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaaaf97-eec6-4dba-be34-419996b5c59d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:56.047608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 39/81] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:08:55 2022] GET /identity/v3/users/57929111b78840da8ec77704273d56d2 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:56.047608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaaaf97-eec6-4dba-be34-419996b5c59d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:56.123253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:56.129883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52aa61-9da8-4e33-8557-3f40bbd10595 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:56.157045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaaaf97-eec6-4dba-be34-419996b5c59d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:56.166311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf6c2b-3928-46d6-a29f-730de8f4f88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:56.167053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf6c2b-3928-46d6-a29f-730de8f4f88a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:56.167343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf6c2b-3928-46d6-a29f-730de8f4f88a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:56.167648 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf6c2b-3928-46d6-a29f-730de8f4f88a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:56.191983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2c3fb-aeb9-4d93-852e-e5f36c0f77ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:56.192740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2c3fb-aeb9-4d93-852e-e5f36c0f77ea None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:56.193125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2c3fb-aeb9-4d93-852e-e5f36c0f77ea None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:56.193501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2c3fb-aeb9-4d93-852e-e5f36c0f77ea None admin] PATH_INFO: `/v3/users/6b6757539b0548b8a38441652f18bce9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:56.194729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2c3fb-aeb9-4d93-852e-e5f36c0f77ea None admin] RBAC: Authorizing `identity:get_user(user_id=6b6757539b0548b8a38441652f18bce9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:56.196260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2c3fb-aeb9-4d93-852e-e5f36c0f77ea None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:56.197846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 40/82] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:08:56 2022] GET /identity/v3/users/6b6757539b0548b8a38441652f18bce9 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:56.198606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2c3fb-aeb9-4d93-852e-e5f36c0f77ea None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:56.216712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:56.237871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf6c2b-3928-46d6-a29f-730de8f4f88a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:56.239366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf6c2b-3928-46d6-a29f-730de8f4f88a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:56.297089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2c3fb-aeb9-4d93-852e-e5f36c0f77ea None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:56.329428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:56.329558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec4b7a-dca1-44a8-899a-df38219a0684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:56.330154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec4b7a-dca1-44a8-899a-df38219a0684 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:56.330779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec4b7a-dca1-44a8-899a-df38219a0684 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:56.331068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec4b7a-dca1-44a8-899a-df38219a0684 None admin] PATH_INFO: `/v3/users/6b6757539b0548b8a38441652f18bce9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:56.332239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec4b7a-dca1-44a8-899a-df38219a0684 None admin] RBAC: Authorizing `identity:get_user(user_id=6b6757539b0548b8a38441652f18bce9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:56.333659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec4b7a-dca1-44a8-899a-df38219a0684 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:56.335717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 41/83] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:08:56 2022] GET /identity/v3/users/6b6757539b0548b8a38441652f18bce9 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:56.336267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec4b7a-dca1-44a8-899a-df38219a0684 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:56.342231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 43/84] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:08:56.342231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf6c2b-3928-46d6-a29f-730de8f4f88a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:56.421548 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:56.436795 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec4b7a-dca1-44a8-899a-df38219a0684 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:56.439025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf6c2b-3928-46d6-a29f-730de8f4f88a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:56.471452 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd91865-8a93-4e89-9432-fc7041f0572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:56.472084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd91865-8a93-4e89-9432-fc7041f0572e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:56.472368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd91865-8a93-4e89-9432-fc7041f0572e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:56.472643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd91865-8a93-4e89-9432-fc7041f0572e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:56.473169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecf843-e265-4d33-a9c8-bd4a1f77d618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:56.473883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecf843-e265-4d33-a9c8-bd4a1f77d618 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:56.474230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecf843-e265-4d33-a9c8-bd4a1f77d618 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:56.474572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecf843-e265-4d33-a9c8-bd4a1f77d618 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:56.475808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecf843-e265-4d33-a9c8-bd4a1f77d618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:56.477303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecf843-e265-4d33-a9c8-bd4a1f77d618 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:56.478871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 44/85] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:08:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:56.479754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecf843-e265-4d33-a9c8-bd4a1f77d618 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:56.540555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd91865-8a93-4e89-9432-fc7041f0572e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:56.541342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:56.542008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd91865-8a93-4e89-9432-fc7041f0572e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:56.580437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecf843-e265-4d33-a9c8-bd4a1f77d618 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:56.614665 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0225d1-f837-4e02-8cb4-de03307be695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:56.615266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0225d1-f837-4e02-8cb4-de03307be695 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:56.615573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0225d1-f837-4e02-8cb4-de03307be695 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:56.615850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0225d1-f837-4e02-8cb4-de03307be695 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:56.623627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:56.639787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 42/86] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:08:56.640616 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd91865-8a93-4e89-9432-fc7041f0572e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:56.684931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0225d1-f837-4e02-8cb4-de03307be695 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:56.686436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0225d1-f837-4e02-8cb4-de03307be695 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:56.738124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd91865-8a93-4e89-9432-fc7041f0572e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:56.746131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:56.771847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466fb2d-48b3-404d-8b15-bea08ec35ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:56.772455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466fb2d-48b3-404d-8b15-bea08ec35ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:56.772730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466fb2d-48b3-404d-8b15-bea08ec35ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:56.772996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466fb2d-48b3-404d-8b15-bea08ec35ac7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:56.785028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 45/87] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:08:56.785028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0225d1-f837-4e02-8cb4-de03307be695 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:56.824607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:56.841695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466fb2d-48b3-404d-8b15-bea08ec35ac7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:56.843139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466fb2d-48b3-404d-8b15-bea08ec35ac7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:56.884189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0225d1-f837-4e02-8cb4-de03307be695 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:56.917548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2faf9-e453-4078-826d-c937e4208865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:56.918170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2faf9-e453-4078-826d-c937e4208865 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:56.918478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2faf9-e453-4078-826d-c937e4208865 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:56.918753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2faf9-e453-4078-826d-c937e4208865 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:56.919893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2faf9-e453-4078-826d-c937e4208865 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:56.921328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2faf9-e453-4078-826d-c937e4208865 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:56.922932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 46/88] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:08:56 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:56.923703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2faf9-e453-4078-826d-c937e4208865 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:56.945163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 43/89] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:08:56.945996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466fb2d-48b3-404d-8b15-bea08ec35ac7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:56.953662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:57.022973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2faf9-e453-4078-826d-c937e4208865 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:57.028207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:57.043599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466fb2d-48b3-404d-8b15-bea08ec35ac7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:57.055993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689141f-9456-4585-ae67-9a9e464e0fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:57.056609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689141f-9456-4585-ae67-9a9e464e0fdf None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:57.056891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689141f-9456-4585-ae67-9a9e464e0fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:57.057169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689141f-9456-4585-ae67-9a9e464e0fdf None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:57.057812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689141f-9456-4585-ae67-9a9e464e0fdf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:57.059849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689141f-9456-4585-ae67-9a9e464e0fdf None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:57.079072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdbc70-7826-4222-a7ba-5818a29c3432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:57.081085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdbc70-7826-4222-a7ba-5818a29c3432 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:57.081085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdbc70-7826-4222-a7ba-5818a29c3432 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:57.081085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdbc70-7826-4222-a7ba-5818a29c3432 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:57.081397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdbc70-7826-4222-a7ba-5818a29c3432 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:57.082811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdbc70-7826-4222-a7ba-5818a29c3432 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:57.084725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 44/90] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:08:56 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:57.085475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdbc70-7826-4222-a7ba-5818a29c3432 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:57.152345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 47/91] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:08:56 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:08:57.153188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689141f-9456-4585-ae67-9a9e464e0fdf None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:57.202629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdbc70-7826-4222-a7ba-5818a29c3432 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:57.248366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:57.249439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd8819-5a80-452e-83c1-1cd131ddeb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:57.250093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8819-5a80-452e-83c1-1cd131ddeb04 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:57.250373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8819-5a80-452e-83c1-1cd131ddeb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:57.250648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8819-5a80-452e-83c1-1cd131ddeb04 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:57.251994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd8819-5a80-452e-83c1-1cd131ddeb04 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:57.253309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd8819-5a80-452e-83c1-1cd131ddeb04 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:57.260209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 45/92] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:08:57 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:57.260209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd8819-5a80-452e-83c1-1cd131ddeb04 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:57.270409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689141f-9456-4585-ae67-9a9e464e0fdf None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:57.283710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:57.314966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4b15c-a7b4-44b2-a53e-1a964d26a394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:57.315662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4b15c-a7b4-44b2-a53e-1a964d26a394 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:57.315951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4b15c-a7b4-44b2-a53e-1a964d26a394 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:57.316230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4b15c-a7b4-44b2-a53e-1a964d26a394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:57.316962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4b15c-a7b4-44b2-a53e-1a964d26a394 None admin] RBAC: Adding query filter params (role.id=0062155b0fbf41a19c9110ad71a046e3, scope.project.id=04853db2b388437e916da478e79b0816, user.id=6b6757539b0548b8a38441652f18bce9) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:57.317298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4b15c-a7b4-44b2-a53e-1a964d26a394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:57.318757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4b15c-a7b4-44b2-a53e-1a964d26a394 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:57.324455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 48/93] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:08:57 2022] GET /identity/v3/role_assignments?user.id=6b6757539b0548b8a38441652f18bce9&scope.project.id=04853db2b388437e916da478e79b0816&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:57.325296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4b15c-a7b4-44b2-a53e-1a964d26a394 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:57.390215 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd8819-5a80-452e-83c1-1cd131ddeb04 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:57.438075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4585954-8baa-434d-ac80-78aed7b52e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:57.438513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4585954-8baa-434d-ac80-78aed7b52e35 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:57.438793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4585954-8baa-434d-ac80-78aed7b52e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:57.439066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4585954-8baa-434d-ac80-78aed7b52e35 None admin] PATH_INFO: `/v3/projects/babc78c64eed4065a5c1558dc4043b4c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:57.440337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4585954-8baa-434d-ac80-78aed7b52e35 None admin] RBAC: Authorizing `identity:get_project(project_id=babc78c64eed4065a5c1558dc4043b4c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:57.441897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4585954-8baa-434d-ac80-78aed7b52e35 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:57.444583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 46/94] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:08:57 2022] GET /identity/v3/projects/babc78c64eed4065a5c1558dc4043b4c => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:57.444583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4585954-8baa-434d-ac80-78aed7b52e35 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:57.484918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4b15c-a7b4-44b2-a53e-1a964d26a394 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:57.538942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320923a-77d3-4155-b932-194aad9453a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:57.539618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320923a-77d3-4155-b932-194aad9453a7 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:57.539905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320923a-77d3-4155-b932-194aad9453a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:57.540180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320923a-77d3-4155-b932-194aad9453a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:57.540960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320923a-77d3-4155-b932-194aad9453a7 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=04853db2b388437e916da478e79b0816, user.id=57929111b78840da8ec77704273d56d2) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:57.541270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320923a-77d3-4155-b932-194aad9453a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:57.542720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320923a-77d3-4155-b932-194aad9453a7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:57.550189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 49/95] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:08:57 2022] GET /identity/v3/role_assignments?user.id=57929111b78840da8ec77704273d56d2&scope.project.id=04853db2b388437e916da478e79b0816&role.id=705907b9bdec47cebf88b868a6a56356 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:57.550189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320923a-77d3-4155-b932-194aad9453a7 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:08:57.580327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:57.582614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:08:57.599824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4585954-8baa-434d-ac80-78aed7b52e35 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:57.648206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169afbf-78ac-41b0-b2b2-970ff30cf6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:57.648854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169afbf-78ac-41b0-b2b2-970ff30cf6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:57.649136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169afbf-78ac-41b0-b2b2-970ff30cf6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:57.649413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169afbf-78ac-41b0-b2b2-970ff30cf6b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:57.650188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169afbf-78ac-41b0-b2b2-970ff30cf6b5 None admin] RBAC: Adding query filter params (role.id=43008618c1dc4cd49109f35c778080d3, scope.project.id=04853db2b388437e916da478e79b0816, user.id=6b6757539b0548b8a38441652f18bce9) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:57.650499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169afbf-78ac-41b0-b2b2-970ff30cf6b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:57.653279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169afbf-78ac-41b0-b2b2-970ff30cf6b5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:57.661655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 47/96] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:08:57 2022] GET /identity/v3/role_assignments?user.id=6b6757539b0548b8a38441652f18bce9&scope.project.id=04853db2b388437e916da478e79b0816&role.id=43008618c1dc4cd49109f35c778080d3 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:57.686237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320923a-77d3-4155-b932-194aad9453a7 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:08:57.732900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a661c-86ca-4e3e-876f-fd8d0ac2d8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:08:57.732900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a661c-86ca-4e3e-876f-fd8d0ac2d8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:57.732900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a661c-86ca-4e3e-876f-fd8d0ac2d8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:57.732900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a661c-86ca-4e3e-876f-fd8d0ac2d8d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:57.733305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a661c-86ca-4e3e-876f-fd8d0ac2d8d0 None admin] RBAC: Adding query filter params (role.id=0062155b0fbf41a19c9110ad71a046e3, scope.project.id=babc78c64eed4065a5c1558dc4043b4c, user.id=6b6757539b0548b8a38441652f18bce9) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:08:57.733609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a661c-86ca-4e3e-876f-fd8d0ac2d8d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:08:57.735407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a661c-86ca-4e3e-876f-fd8d0ac2d8d0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:08:57.742258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 50/97] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:08:57 2022] GET /identity/v3/role_assignments?user.id=6b6757539b0548b8a38441652f18bce9&scope.project.id=babc78c64eed4065a5c1558dc4043b4c&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:08:59.889527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e22f58-9908-4a53-bcff-f4caf0467974 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:59.890054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e22f58-9908-4a53-bcff-f4caf0467974 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:59.890054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e22f58-9908-4a53-bcff-f4caf0467974 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:59.891580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 48/98] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:08:59.896983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9753de-fee5-412e-96d5-1f0ff12becfa None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:08:59.897375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9753de-fee5-412e-96d5-1f0ff12becfa None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:08:59.897535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9753de-fee5-412e-96d5-1f0ff12becfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:08:59.920157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-0d9753de-fee5-412e-96d5-1f0ff12becfa None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:08:59.921364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0d9753de-fee5-412e-96d5-1f0ff12becfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:00.001377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 51/99] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:00.131071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b15caa-a18e-42c7-aea6-6758471b2851 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:00.131408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b15caa-a18e-42c7-aea6-6758471b2851 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:00.131655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b15caa-a18e-42c7-aea6-6758471b2851 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:00.133235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 49/100] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:00.139207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f002b8-2aff-41e0-bef8-bcdf09794ce6 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:00.139497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f002b8-2aff-41e0-bef8-bcdf09794ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:00.139864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f002b8-2aff-41e0-bef8-bcdf09794ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:00.158648 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-70f002b8-2aff-41e0-bef8-bcdf09794ce6 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:00.159875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-70f002b8-2aff-41e0-bef8-bcdf09794ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:00.161936 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:00.252421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 52/101] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:09:00.372134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47c36a-467e-4605-8d71-c6b1622f640c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:00.372464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47c36a-467e-4605-8d71-c6b1622f640c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:00.372694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47c36a-467e-4605-8d71-c6b1622f640c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:00.374046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 50/102] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:00.385118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544138f-92a4-4f86-b02a-7d15b93162ed None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:00.385409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544138f-92a4-4f86-b02a-7d15b93162ed None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:00.385691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544138f-92a4-4f86-b02a-7d15b93162ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:00.400549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11bc5e-0286-4191-ac90-cab58e33f7ac None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:00.400549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11bc5e-0286-4191-ac90-cab58e33f7ac None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:00.400549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11bc5e-0286-4191-ac90-cab58e33f7ac None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:00.401916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 51/103] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:09:00.411993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-5544138f-92a4-4f86-b02a-7d15b93162ed None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:00.413258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5544138f-92a4-4f86-b02a-7d15b93162ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:00.418114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fd40-0d64-4e3a-8c69-dfdc2b9b04fd None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:00.418402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fd40-0d64-4e3a-8c69-dfdc2b9b04fd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:00.418731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fd40-0d64-4e3a-8c69-dfdc2b9b04fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:00.451920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0b44fd40-0d64-4e3a-8c69-dfdc2b9b04fd None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:00.453167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0b44fd40-0d64-4e3a-8c69-dfdc2b9b04fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:00.549144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 52/104] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:09:00.550520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b0166-84a2-4106-96c7-edb826f4187d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:00.550870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b0166-84a2-4106-96c7-edb826f4187d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:00.551146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b0166-84a2-4106-96c7-edb826f4187d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:00.552430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 53/105] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:00.553154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b0166-84a2-4106-96c7-edb826f4187d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:00.592967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 53/106] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:09:00.595618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29258c9-67a1-4741-8bef-ecfbdb16e992 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:00.595908 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29258c9-67a1-4741-8bef-ecfbdb16e992 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:00.596184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29258c9-67a1-4741-8bef-ecfbdb16e992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:00.631350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c29258c9-67a1-4741-8bef-ecfbdb16e992 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:00.634143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c29258c9-67a1-4741-8bef-ecfbdb16e992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:00.719972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:00.729302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:00.738778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 54/107] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:00.741314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29258c9-67a1-4741-8bef-ecfbdb16e992 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:00.751638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b0166-84a2-4106-96c7-edb826f4187d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:00.819166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d95d7-6a03-49f0-9956-4012ad9a87ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:00.820115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d95d7-6a03-49f0-9956-4012ad9a87ce None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:00.820565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d95d7-6a03-49f0-9956-4012ad9a87ce None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:00.821005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d95d7-6a03-49f0-9956-4012ad9a87ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:00.822728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d95d7-6a03-49f0-9956-4012ad9a87ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:00.824439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d95d7-6a03-49f0-9956-4012ad9a87ce None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:00.835015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 54/108] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:00.837347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d95d7-6a03-49f0-9956-4012ad9a87ce None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:00.912853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29258c9-67a1-4741-8bef-ecfbdb16e992 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:00.948948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a8c86-a215-42e9-aefb-cf4903326336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:00.950593 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a8c86-a215-42e9-aefb-cf4903326336 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:00.951045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a8c86-a215-42e9-aefb-cf4903326336 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:00.951517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a8c86-a215-42e9-aefb-cf4903326336 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:00.962526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a8c86-a215-42e9-aefb-cf4903326336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-3d8a8c86-a215-42e9-aefb-cf4903326336 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:00.967619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:00.968464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:00.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:00.969885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d95d7-6a03-49f0-9956-4012ad9a87ce None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:00.975689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 55/109] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:00.976691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a8c86-a215-42e9-aefb-cf4903326336 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.006280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1c6f2-4f5e-4a6b-8fad-6495c1ea0c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:01.006904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1c6f2-4f5e-4a6b-8fad-6495c1ea0c59 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.007171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1c6f2-4f5e-4a6b-8fad-6495c1ea0c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.007445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1c6f2-4f5e-4a6b-8fad-6495c1ea0c59 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.008197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1c6f2-4f5e-4a6b-8fad-6495c1ea0c59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:01.009944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1c6f2-4f5e-4a6b-8fad-6495c1ea0c59 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:01.040590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:01.085678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a8c86-a215-42e9-aefb-cf4903326336 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:01.111569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 55/110] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:09:00 2022] POST /identity/v3/projects => generated 302 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:09:01.111569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1c6f2-4f5e-4a6b-8fad-6495c1ea0c59 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.111894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:01.135072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034fc53-d55c-44e5-aefa-729385f1639f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:01.135772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034fc53-d55c-44e5-aefa-729385f1639f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.136078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034fc53-d55c-44e5-aefa-729385f1639f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.136339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034fc53-d55c-44e5-aefa-729385f1639f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.149266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034fc53-d55c-44e5-aefa-729385f1639f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-4034fc53-d55c-44e5-aefa-729385f1639f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:01.154437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:01.155519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:01.156463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 56/111] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:01.161468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034fc53-d55c-44e5-aefa-729385f1639f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.237880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1c6f2-4f5e-4a6b-8fad-6495c1ea0c59 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:01.274109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034fc53-d55c-44e5-aefa-729385f1639f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:01.277569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7cd75-5d39-4771-90db-29c528fb9e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:01.278213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7cd75-5d39-4771-90db-29c528fb9e66 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.278491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7cd75-5d39-4771-90db-29c528fb9e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.278761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7cd75-5d39-4771-90db-29c528fb9e66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.290556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7cd75-5d39-4771-90db-29c528fb9e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:01.294083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-83e7cd75-5d39-4771-90db-29c528fb9e66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:01.297444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:01.298298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:01.299065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 56/112] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:01.300344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e64bdd-6ebb-4849-bfae-2444135e5315 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.300344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e64bdd-6ebb-4849-bfae-2444135e5315 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.300344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e64bdd-6ebb-4849-bfae-2444135e5315 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.301504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 57/113] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:01.302235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e64bdd-6ebb-4849-bfae-2444135e5315 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.315280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8701def-8f6d-481b-a53d-87d0c27844e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:01.316308 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8701def-8f6d-481b-a53d-87d0c27844e3 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.316868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8701def-8f6d-481b-a53d-87d0c27844e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.316868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8701def-8f6d-481b-a53d-87d0c27844e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.317596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8701def-8f6d-481b-a53d-87d0c27844e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:01.317897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8701def-8f6d-481b-a53d-87d0c27844e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:01.319281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8701def-8f6d-481b-a53d-87d0c27844e3 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:01.328294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 57/114] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:01.333603 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8701def-8f6d-481b-a53d-87d0c27844e3 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.372434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:01.423550 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e64bdd-6ebb-4849-bfae-2444135e5315 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:01.445572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8701def-8f6d-481b-a53d-87d0c27844e3 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:01.459630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52077d30-fc63-4c27-ac78-87ae4b4f2f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:01.460264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52077d30-fc63-4c27-ac78-87ae4b4f2f5f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.460543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52077d30-fc63-4c27-ac78-87ae4b4f2f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.460814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52077d30-fc63-4c27-ac78-87ae4b4f2f5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.471391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52077d30-fc63-4c27-ac78-87ae4b4f2f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-52077d30-fc63-4c27-ac78-87ae4b4f2f5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:01.476931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:01.477818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 58/115] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:01.480440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52077d30-fc63-4c27-ac78-87ae4b4f2f5f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.494916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c8844-a6f5-4637-a493-daee79313529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:01.495470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c8844-a6f5-4637-a493-daee79313529 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.495794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c8844-a6f5-4637-a493-daee79313529 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.496096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c8844-a6f5-4637-a493-daee79313529 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.497204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c8844-a6f5-4637-a493-daee79313529 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:01.497526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c8844-a6f5-4637-a493-daee79313529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:01.498937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c8844-a6f5-4637-a493-daee79313529 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:01.507855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 58/116] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:01.513398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32097530-d1a2-46f3-a06b-d4f203728464 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.513699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32097530-d1a2-46f3-a06b-d4f203728464 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.513984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32097530-d1a2-46f3-a06b-d4f203728464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.532118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-32097530-d1a2-46f3-a06b-d4f203728464 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:01.533325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-32097530-d1a2-46f3-a06b-d4f203728464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:01.570640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:01.587915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52077d30-fc63-4c27-ac78-87ae4b4f2f5f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:01.605496 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 59/117] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:01.606942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32097530-d1a2-46f3-a06b-d4f203728464 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.623760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:01.644196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d70baa-bbc5-40db-a344-c1767fac5606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:01.644816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d70baa-bbc5-40db-a344-c1767fac5606 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.645504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d70baa-bbc5-40db-a344-c1767fac5606 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.645838 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d70baa-bbc5-40db-a344-c1767fac5606 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.646905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d70baa-bbc5-40db-a344-c1767fac5606 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:01.647258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d70baa-bbc5-40db-a344-c1767fac5606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:01.648991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d70baa-bbc5-40db-a344-c1767fac5606 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:01.662296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 59/118] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:01.663839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d70baa-bbc5-40db-a344-c1767fac5606 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.735501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32097530-d1a2-46f3-a06b-d4f203728464 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:01.774503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd87662-2243-4120-9509-c342d5378df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:01.775629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd87662-2243-4120-9509-c342d5378df1 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.776199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd87662-2243-4120-9509-c342d5378df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.776686 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd87662-2243-4120-9509-c342d5378df1 None admin] PATH_INFO: `/v3/users/6b6757539b0548b8a38441652f18bce9` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.778680 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd87662-2243-4120-9509-c342d5378df1 None admin] RBAC: Authorizing `identity:get_user(user_id=6b6757539b0548b8a38441652f18bce9)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:01.780607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd87662-2243-4120-9509-c342d5378df1 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:01.782714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 60/119] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/users/6b6757539b0548b8a38441652f18bce9 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:01.783725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd87662-2243-4120-9509-c342d5378df1 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.803515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d70baa-bbc5-40db-a344-c1767fac5606 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:01.831983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:01.848562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92da76e-3770-4500-adc0-ea489ca959eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:01.849174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92da76e-3770-4500-adc0-ea489ca959eb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.849455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92da76e-3770-4500-adc0-ea489ca959eb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.849729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92da76e-3770-4500-adc0-ea489ca959eb None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.850648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92da76e-3770-4500-adc0-ea489ca959eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:01.850950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92da76e-3770-4500-adc0-ea489ca959eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:01.852993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92da76e-3770-4500-adc0-ea489ca959eb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:01.863553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 60/120] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:01.865243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92da76e-3770-4500-adc0-ea489ca959eb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.898943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd87662-2243-4120-9509-c342d5378df1 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:01.924023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:01.934051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fa86a-276d-465e-8c1a-79e4ce858d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:01.934717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fa86a-276d-465e-8c1a-79e4ce858d55 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:01.934996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fa86a-276d-465e-8c1a-79e4ce858d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:01.935270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fa86a-276d-465e-8c1a-79e4ce858d55 None admin] PATH_INFO: `/v3/users/6b6757539b0548b8a38441652f18bce9` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:01.936427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fa86a-276d-465e-8c1a-79e4ce858d55 None admin] RBAC: Authorizing `identity:get_user(user_id=6b6757539b0548b8a38441652f18bce9)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:01.937867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fa86a-276d-465e-8c1a-79e4ce858d55 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:01.944085 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 61/121] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/users/6b6757539b0548b8a38441652f18bce9 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:01.944085 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fa86a-276d-465e-8c1a-79e4ce858d55 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:01.985100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92da76e-3770-4500-adc0-ea489ca959eb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.021216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c75a4-28c7-4e50-ad9d-e1042e6716f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.021850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c75a4-28c7-4e50-ad9d-e1042e6716f4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.022131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c75a4-28c7-4e50-ad9d-e1042e6716f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.022407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c75a4-28c7-4e50-ad9d-e1042e6716f4 None admin] PATH_INFO: `/v3/users/6b6757539b0548b8a38441652f18bce9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.023558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c75a4-28c7-4e50-ad9d-e1042e6716f4 None admin] RBAC: Authorizing `identity:get_user(user_id=6b6757539b0548b8a38441652f18bce9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.025074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c75a4-28c7-4e50-ad9d-e1042e6716f4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.026774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 61/122] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/users/6b6757539b0548b8a38441652f18bce9 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:02.027346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c75a4-28c7-4e50-ad9d-e1042e6716f4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.057737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fa86a-276d-465e-8c1a-79e4ce858d55 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.070414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:02.096251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e33a7-84b2-4c02-b8b8-79e482798921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.096889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e33a7-84b2-4c02-b8b8-79e482798921 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.097165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e33a7-84b2-4c02-b8b8-79e482798921 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.097434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e33a7-84b2-4c02-b8b8-79e482798921 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.098547 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e33a7-84b2-4c02-b8b8-79e482798921 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.100017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e33a7-84b2-4c02-b8b8-79e482798921 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.101815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 62/123] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:01 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:02.102478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e33a7-84b2-4c02-b8b8-79e482798921 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.139887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c75a4-28c7-4e50-ad9d-e1042e6716f4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.155669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:02.175336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1d936-f223-47e4-ad2d-340c8f9717b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.175996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1d936-f223-47e4-ad2d-340c8f9717b7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.176272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1d936-f223-47e4-ad2d-340c8f9717b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.176866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1d936-f223-47e4-ad2d-340c8f9717b7 None admin] PATH_INFO: `/v3/users/3015f5ce64524d2da234cc0cc671e2b5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.191471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1d936-f223-47e4-ad2d-340c8f9717b7 None admin] RBAC: Authorizing `identity:get_user(user_id=3015f5ce64524d2da234cc0cc671e2b5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.192971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1d936-f223-47e4-ad2d-340c8f9717b7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.199164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 62/124] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:02 2022] GET /identity/v3/users/3015f5ce64524d2da234cc0cc671e2b5 => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:02.199164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1d936-f223-47e4-ad2d-340c8f9717b7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.219339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e33a7-84b2-4c02-b8b8-79e482798921 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.254895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7374c5f-7328-4d64-9851-8294a6f7f059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.255567 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7374c5f-7328-4d64-9851-8294a6f7f059 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.255864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7374c5f-7328-4d64-9851-8294a6f7f059 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.256136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7374c5f-7328-4d64-9851-8294a6f7f059 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.266716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7374c5f-7328-4d64-9851-8294a6f7f059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-b7374c5f-7328-4d64-9851-8294a6f7f059 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:02.272102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:02.272980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:02.273743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 63/125] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:02.274372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7374c5f-7328-4d64-9851-8294a6f7f059 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.315770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:02.316396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1d936-f223-47e4-ad2d-340c8f9717b7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.350793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab61c2-d504-4337-a1f0-23f1258b3a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.351428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab61c2-d504-4337-a1f0-23f1258b3a27 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.351746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab61c2-d504-4337-a1f0-23f1258b3a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.352033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab61c2-d504-4337-a1f0-23f1258b3a27 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.353168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab61c2-d504-4337-a1f0-23f1258b3a27 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.354613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab61c2-d504-4337-a1f0-23f1258b3a27 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.356327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 63/126] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:02 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:02.361680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab61c2-d504-4337-a1f0-23f1258b3a27 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.385632 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7374c5f-7328-4d64-9851-8294a6f7f059 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.404373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:02.422099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b9a2a-8e2f-4f8c-bd38-9e548973e819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.422739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b9a2a-8e2f-4f8c-bd38-9e548973e819 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.423017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b9a2a-8e2f-4f8c-bd38-9e548973e819 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.423346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b9a2a-8e2f-4f8c-bd38-9e548973e819 None admin] PATH_INFO: `/v3/projects/babc78c64eed4065a5c1558dc4043b4c` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.424577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b9a2a-8e2f-4f8c-bd38-9e548973e819 None admin] RBAC: Authorizing `identity:get_project(project_id=babc78c64eed4065a5c1558dc4043b4c)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.426028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b9a2a-8e2f-4f8c-bd38-9e548973e819 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.427658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 64/127] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:02 2022] GET /identity/v3/projects/babc78c64eed4065a5c1558dc4043b4c => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:02.428423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b9a2a-8e2f-4f8c-bd38-9e548973e819 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.473310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab61c2-d504-4337-a1f0-23f1258b3a27 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.509547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f8ff0-2a82-4e34-9033-c65487f1307f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.510214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f8ff0-2a82-4e34-9033-c65487f1307f None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.510499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f8ff0-2a82-4e34-9033-c65487f1307f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.510779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f8ff0-2a82-4e34-9033-c65487f1307f None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816/users/6b6757539b0548b8a38441652f18bce9/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.512743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f8ff0-2a82-4e34-9033-c65487f1307f None admin] RBAC: Authorizing `identity:create_grant(project_id=04853db2b388437e916da478e79b0816, user_id=6b6757539b0548b8a38441652f18bce9, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.514225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f8ff0-2a82-4e34-9033-c65487f1307f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.536886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b9a2a-8e2f-4f8c-bd38-9e548973e819 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.539190 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:02.570821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f277635-3187-4f51-b180-83a122e3ef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.571360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f277635-3187-4f51-b180-83a122e3ef7a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.571681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f277635-3187-4f51-b180-83a122e3ef7a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.571952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f277635-3187-4f51-b180-83a122e3ef7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.587791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 64/128] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:02 2022] PUT /identity/v3/projects/04853db2b388437e916da478e79b0816/users/6b6757539b0548b8a38441652f18bce9/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:02.588956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f8ff0-2a82-4e34-9033-c65487f1307f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.676481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f277635-3187-4f51-b180-83a122e3ef7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.679168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f277635-3187-4f51-b180-83a122e3ef7a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.730355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:02.737072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f8ff0-2a82-4e34-9033-c65487f1307f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.783815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60ba41-f169-455e-a424-3fbd97569a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.784430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60ba41-f169-455e-a424-3fbd97569a60 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.784707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60ba41-f169-455e-a424-3fbd97569a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.784980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60ba41-f169-455e-a424-3fbd97569a60 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.786048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60ba41-f169-455e-a424-3fbd97569a60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:02.786352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60ba41-f169-455e-a424-3fbd97569a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.788117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60ba41-f169-455e-a424-3fbd97569a60 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.798118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 65/129] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:02.799172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60ba41-f169-455e-a424-3fbd97569a60 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.811754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 65/130] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:02.812591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f277635-3187-4f51-b180-83a122e3ef7a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:02.816668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:02.926515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f277635-3187-4f51-b180-83a122e3ef7a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.931308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60ba41-f169-455e-a424-3fbd97569a60 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:02.962750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d8bbc-2261-4784-89ba-4a123fcbf931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.963417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8bbc-2261-4784-89ba-4a123fcbf931 None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.963738 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8bbc-2261-4784-89ba-4a123fcbf931 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.964012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8bbc-2261-4784-89ba-4a123fcbf931 None admin] PATH_INFO: `/v3/projects/babc78c64eed4065a5c1558dc4043b4c/users/6b6757539b0548b8a38441652f18bce9/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.966844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d8bbc-2261-4784-89ba-4a123fcbf931 None admin] RBAC: Authorizing `identity:create_grant(project_id=babc78c64eed4065a5c1558dc4043b4c, user_id=6b6757539b0548b8a38441652f18bce9, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.969109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d8bbc-2261-4784-89ba-4a123fcbf931 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:02.975137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d07690-57ed-4fe4-90f7-e19bdbb78708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:02.975913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07690-57ed-4fe4-90f7-e19bdbb78708 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:02.976190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07690-57ed-4fe4-90f7-e19bdbb78708 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:02.976459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07690-57ed-4fe4-90f7-e19bdbb78708 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816/users/6b6757539b0548b8a38441652f18bce9/roles/43008618c1dc4cd49109f35c778080d3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:02.978700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d07690-57ed-4fe4-90f7-e19bdbb78708 None admin] RBAC: Authorizing `identity:create_grant(project_id=04853db2b388437e916da478e79b0816, user_id=6b6757539b0548b8a38441652f18bce9, role_id=43008618c1dc4cd49109f35c778080d3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:02.980253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d07690-57ed-4fe4-90f7-e19bdbb78708 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:03.073314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 66/131] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:02 2022] PUT /identity/v3/projects/babc78c64eed4065a5c1558dc4043b4c/users/6b6757539b0548b8a38441652f18bce9/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:03.074348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d8bbc-2261-4784-89ba-4a123fcbf931 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:03.109670 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 66/132] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:02 2022] PUT /identity/v3/projects/04853db2b388437e916da478e79b0816/users/6b6757539b0548b8a38441652f18bce9/roles/43008618c1dc4cd49109f35c778080d3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:03.110885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d07690-57ed-4fe4-90f7-e19bdbb78708 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:03.172100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:03.198042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:03.256282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d8bbc-2261-4784-89ba-4a123fcbf931 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:03.305714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fdf54-4b9c-4c51-8b32-fc1869f58857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:03.307412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fdf54-4b9c-4c51-8b32-fc1869f58857 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.307895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fdf54-4b9c-4c51-8b32-fc1869f58857 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.308607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fdf54-4b9c-4c51-8b32-fc1869f58857 None admin] PATH_INFO: `/v3/users/57929111b78840da8ec77704273d56d2` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.310792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fdf54-4b9c-4c51-8b32-fc1869f58857 None admin] RBAC: Authorizing `identity:get_user(user_id=57929111b78840da8ec77704273d56d2)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:03.313131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fdf54-4b9c-4c51-8b32-fc1869f58857 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:03.315573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 67/133] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:03 2022] GET /identity/v3/users/57929111b78840da8ec77704273d56d2 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:03.316076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d07690-57ed-4fe4-90f7-e19bdbb78708 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:03.317127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca240d-bcbf-4871-b43f-056349a8e707 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.317862 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca240d-bcbf-4871-b43f-056349a8e707 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.318372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca240d-bcbf-4871-b43f-056349a8e707 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.321086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 68/134] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:03.321981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca240d-bcbf-4871-b43f-056349a8e707 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:03.360287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe94e46-bb6d-4650-a480-581ab99d924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:03.360906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe94e46-bb6d-4650-a480-581ab99d924f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.361189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe94e46-bb6d-4650-a480-581ab99d924f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.361461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe94e46-bb6d-4650-a480-581ab99d924f None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.362699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe94e46-bb6d-4650-a480-581ab99d924f None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:03.364983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe94e46-bb6d-4650-a480-581ab99d924f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:03.367047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 67/135] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:03 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:03.371569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784237f6-8c32-440c-8a2b-8a89b62db25b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.371569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784237f6-8c32-440c-8a2b-8a89b62db25b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.371569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784237f6-8c32-440c-8a2b-8a89b62db25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.392199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-784237f6-8c32-440c-8a2b-8a89b62db25b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:03.393669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-784237f6-8c32-440c-8a2b-8a89b62db25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:03.461814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca240d-bcbf-4871-b43f-056349a8e707 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:03.483627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 68/136] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:03.484722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784237f6-8c32-440c-8a2b-8a89b62db25b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:03.524767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:03.542171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec856c2-d352-4376-93bd-c5507b30b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:03.543919 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec856c2-d352-4376-93bd-c5507b30b1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.547410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec856c2-d352-4376-93bd-c5507b30b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.547928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec856c2-d352-4376-93bd-c5507b30b1a2 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.549279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec856c2-d352-4376-93bd-c5507b30b1a2 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:03.550870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec856c2-d352-4376-93bd-c5507b30b1a2 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:03.551282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:03.554925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 69/137] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:03 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:03.567758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec856c2-d352-4376-93bd-c5507b30b1a2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:03.705291 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec856c2-d352-4376-93bd-c5507b30b1a2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:03.723507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784237f6-8c32-440c-8a2b-8a89b62db25b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:03.741867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3247d6-7879-4021-8837-39c46d8615d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:03.742533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3247d6-7879-4021-8837-39c46d8615d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.742817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3247d6-7879-4021-8837-39c46d8615d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.743098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3247d6-7879-4021-8837-39c46d8615d2 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816/users/57929111b78840da8ec77704273d56d2/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.745129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3247d6-7879-4021-8837-39c46d8615d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=04853db2b388437e916da478e79b0816, user_id=57929111b78840da8ec77704273d56d2, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:03.747227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3247d6-7879-4021-8837-39c46d8615d2 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:03.798321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b70045-6fc0-4b6f-9505-d0e153928b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:03.798940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b70045-6fc0-4b6f-9505-d0e153928b5b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.799219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b70045-6fc0-4b6f-9505-d0e153928b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.799492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b70045-6fc0-4b6f-9505-d0e153928b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.804474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b70045-6fc0-4b6f-9505-d0e153928b5b None admin] RBAC: Adding query filter params (role.id=b19408717eaf492c9999c46062b83ba8, scope.project.id=04853db2b388437e916da478e79b0816, user.id=3015f5ce64524d2da234cc0cc671e2b5) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:03.804802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b70045-6fc0-4b6f-9505-d0e153928b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:03.806355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b70045-6fc0-4b6f-9505-d0e153928b5b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:03.813066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 69/138] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:03 2022] GET /identity/v3/role_assignments?user.id=3015f5ce64524d2da234cc0cc671e2b5&scope.project.id=04853db2b388437e916da478e79b0816&role.id=b19408717eaf492c9999c46062b83ba8 => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:03.813867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b70045-6fc0-4b6f-9505-d0e153928b5b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:03.828651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 70/139] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:03 2022] PUT /identity/v3/projects/04853db2b388437e916da478e79b0816/users/57929111b78840da8ec77704273d56d2/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:03.914473 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:03.933030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b70045-6fc0-4b6f-9505-d0e153928b5b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:03.974265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093796c-dce4-41bf-aac5-45f10ee09ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:03.974849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093796c-dce4-41bf-aac5-45f10ee09ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:03.975136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093796c-dce4-41bf-aac5-45f10ee09ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:03.975411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093796c-dce4-41bf-aac5-45f10ee09ec7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:03.976613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093796c-dce4-41bf-aac5-45f10ee09ec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:03.978167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093796c-dce4-41bf-aac5-45f10ee09ec7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:03.980079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 70/140] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:03.986414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3247d6-7879-4021-8837-39c46d8615d2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:04.109183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:04.154264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3247d6-7879-4021-8837-39c46d8615d2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:04.190433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0cebd-72b9-45d8-8cc9-cb336524b0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:04.191122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0cebd-72b9-45d8-8cc9-cb336524b0b8 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:04.191414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0cebd-72b9-45d8-8cc9-cb336524b0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:04.191724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0cebd-72b9-45d8-8cc9-cb336524b0b8 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:04.192375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0cebd-72b9-45d8-8cc9-cb336524b0b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:04.193839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0cebd-72b9-45d8-8cc9-cb336524b0b8 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:04.314717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 71/141] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:09:03 2022] POST /identity/v3/users => generated 283 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:05.199367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137681a8-5285-40fd-9c6e-bba60964bc7d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:05.199968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137681a8-5285-40fd-9c6e-bba60964bc7d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:05.200042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137681a8-5285-40fd-9c6e-bba60964bc7d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:05.201646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 71/142] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:05.206763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e1bcb-fca2-4c0c-bd0e-a20998123de3 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:05.207163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e1bcb-fca2-4c0c-bd0e-a20998123de3 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:05.207350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e1bcb-fca2-4c0c-bd0e-a20998123de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:05.236560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-593e1bcb-fca2-4c0c-bd0e-a20998123de3 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:05.237789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-593e1bcb-fca2-4c0c-bd0e-a20998123de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:05.321842 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 72/143] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:05.497729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43801fb1-3ead-460e-9801-03f8c774f261 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:05.498060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43801fb1-3ead-460e-9801-03f8c774f261 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:05.498287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43801fb1-3ead-460e-9801-03f8c774f261 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:05.499765 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 72/144] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:05.512097 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2764-5946-46fd-a845-580dce44cf11 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:05.512991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2764-5946-46fd-a845-580dce44cf11 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:05.513328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2764-5946-46fd-a845-580dce44cf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:05.540575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d0ed2764-5946-46fd-a845-580dce44cf11 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:05.541798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d0ed2764-5946-46fd-a845-580dce44cf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:05.595302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:05.626337 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 73/145] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:05.737046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43801fb1-3ead-460e-9801-03f8c774f261 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:05.872926 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43801fb1-3ead-460e-9801-03f8c774f261 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:05.916781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87938bfd-4b1a-4d84-94b6-ba767ad8c12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:05.917415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87938bfd-4b1a-4d84-94b6-ba767ad8c12b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:05.917693 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87938bfd-4b1a-4d84-94b6-ba767ad8c12b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:05.917965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87938bfd-4b1a-4d84-94b6-ba767ad8c12b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:05.932941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87938bfd-4b1a-4d84-94b6-ba767ad8c12b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-87938bfd-4b1a-4d84-94b6-ba767ad8c12b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:05.938821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:05.939694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:05.940438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:05.941477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 73/146] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:05.948080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed2764-5946-46fd-a845-580dce44cf11 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:06.081935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed2764-5946-46fd-a845-580dce44cf11 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:06.103557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87938bfd-4b1a-4d84-94b6-ba767ad8c12b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:06.122702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:06.123814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257191-eb66-4acc-8946-253177d3149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:06.124386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257191-eb66-4acc-8946-253177d3149b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:06.124666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257191-eb66-4acc-8946-253177d3149b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:06.124944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257191-eb66-4acc-8946-253177d3149b None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:06.125679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a257191-eb66-4acc-8946-253177d3149b None admin] RBAC: Adding query filter params (name=member) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:06.125986 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a257191-eb66-4acc-8946-253177d3149b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:06.127457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a257191-eb66-4acc-8946-253177d3149b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:06.140659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 74/147] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:06.152657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257191-eb66-4acc-8946-253177d3149b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:06.211748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:06.254217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87938bfd-4b1a-4d84-94b6-ba767ad8c12b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:06.303045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83faff91-a0d9-4104-8186-77cb28e52fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:06.303714 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83faff91-a0d9-4104-8186-77cb28e52fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:06.303990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83faff91-a0d9-4104-8186-77cb28e52fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:06.304257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83faff91-a0d9-4104-8186-77cb28e52fd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:06.318367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257191-eb66-4acc-8946-253177d3149b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:06.325563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83faff91-a0d9-4104-8186-77cb28e52fd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-83faff91-a0d9-4104-8186-77cb28e52fd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:06.330886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:06.331752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:06.332579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:06.340862 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 74/148] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:06.343967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998dec-1728-4670-a10a-b3739e33512d None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:06.344247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998dec-1728-4670-a10a-b3739e33512d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:06.344517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998dec-1728-4670-a10a-b3739e33512d None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:06.345812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 75/149] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:06.351426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998dec-1728-4670-a10a-b3739e33512d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:06.384354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da914b2e-4464-4d88-929c-e43a71e4cf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:06.385301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da914b2e-4464-4d88-929c-e43a71e4cf0f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:06.385780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da914b2e-4464-4d88-929c-e43a71e4cf0f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:06.386187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da914b2e-4464-4d88-929c-e43a71e4cf0f None admin] PATH_INFO: `/v3/users/6b6757539b0548b8a38441652f18bce9` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:06.387647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da914b2e-4464-4d88-929c-e43a71e4cf0f None admin] RBAC: Authorizing `identity:get_user(user_id=6b6757539b0548b8a38441652f18bce9)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:06.389251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da914b2e-4464-4d88-929c-e43a71e4cf0f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:06.391271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 75/150] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:06 2022] GET /identity/v3/users/6b6757539b0548b8a38441652f18bce9 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:06.393023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b293d8b-03dd-4c96-8605-c828de8971c2 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:06.399263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b293d8b-03dd-4c96-8605-c828de8971c2 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:06.399263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b293d8b-03dd-4c96-8605-c828de8971c2 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:06.399263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 76/151] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:06.399263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae17e37-9d99-4274-b0b7-db313ef80a80 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:06.399263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae17e37-9d99-4274-b0b7-db313ef80a80 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:06.399263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae17e37-9d99-4274-b0b7-db313ef80a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:06.418309 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7ae17e37-9d99-4274-b0b7-db313ef80a80 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:06.420104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7ae17e37-9d99-4274-b0b7-db313ef80a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:06.455529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:06.509309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998dec-1728-4670-a10a-b3739e33512d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:06.528796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 77/152] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:06.530110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab90cb-fca5-4d15-9e2a-0f170bb8afc0 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:06.530395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab90cb-fca5-4d15-9e2a-0f170bb8afc0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:06.530666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab90cb-fca5-4d15-9e2a-0f170bb8afc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:06.561740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-86ab90cb-fca5-4d15-9e2a-0f170bb8afc0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:06.562984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-86ab90cb-fca5-4d15-9e2a-0f170bb8afc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:06.576344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b70de8-b9b6-4a0a-8ce1-7f2205e58c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:06.577429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b70de8-b9b6-4a0a-8ce1-7f2205e58c11 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:06.577916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b70de8-b9b6-4a0a-8ce1-7f2205e58c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:06.578736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b70de8-b9b6-4a0a-8ce1-7f2205e58c11 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:06.579929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b70de8-b9b6-4a0a-8ce1-7f2205e58c11 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:06.580427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b70de8-b9b6-4a0a-8ce1-7f2205e58c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:06.582504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b70de8-b9b6-4a0a-8ce1-7f2205e58c11 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:06.600496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 76/153] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:06.602641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b70de8-b9b6-4a0a-8ce1-7f2205e58c11 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:06.604421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:06.679144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 78/154] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:06.680017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab90cb-fca5-4d15-9e2a-0f170bb8afc0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:06.782659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b70de8-b9b6-4a0a-8ce1-7f2205e58c11 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:06.836994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da22e6e-9257-4a45-a62e-68ab2cd0f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:06.837652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da22e6e-9257-4a45-a62e-68ab2cd0f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:06.837879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da22e6e-9257-4a45-a62e-68ab2cd0f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:06.838148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da22e6e-9257-4a45-a62e-68ab2cd0f6c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:06.849705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab90cb-fca5-4d15-9e2a-0f170bb8afc0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:06.855868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:06.917261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da22e6e-9257-4a45-a62e-68ab2cd0f6c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:06.919499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da22e6e-9257-4a45-a62e-68ab2cd0f6c5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:06.948904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863ff7d-376c-4291-b24e-0c910c904de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:06.949479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863ff7d-376c-4291-b24e-0c910c904de2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:06.949789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863ff7d-376c-4291-b24e-0c910c904de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:06.950066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863ff7d-376c-4291-b24e-0c910c904de2 None admin] PATH_INFO: `/v3/users/6b6757539b0548b8a38441652f18bce9` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:06.956303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863ff7d-376c-4291-b24e-0c910c904de2 None admin] RBAC: Authorizing `identity:get_user(user_id=6b6757539b0548b8a38441652f18bce9)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:06.957784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863ff7d-376c-4291-b24e-0c910c904de2 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:06.960011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 79/155] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:06 2022] GET /identity/v3/users/6b6757539b0548b8a38441652f18bce9 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:06.965128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863ff7d-376c-4291-b24e-0c910c904de2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.025606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 77/156] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:07.031612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da22e6e-9257-4a45-a62e-68ab2cd0f6c5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.064956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:07.073240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:07.143458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da22e6e-9257-4a45-a62e-68ab2cd0f6c5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:07.180564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863ff7d-376c-4291-b24e-0c910c904de2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:07.196329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e827a14-807a-47c7-9127-28e015af44bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:07.196975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e827a14-807a-47c7-9127-28e015af44bd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.197245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e827a14-807a-47c7-9127-28e015af44bd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.197512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e827a14-807a-47c7-9127-28e015af44bd None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.198768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e827a14-807a-47c7-9127-28e015af44bd None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:07.200969 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e827a14-807a-47c7-9127-28e015af44bd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:07.202891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 78/157] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:07.208884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e827a14-807a-47c7-9127-28e015af44bd None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.237339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb379f83-ad88-47e0-ae0f-f102d1a4a90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:07.237984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb379f83-ad88-47e0-ae0f-f102d1a4a90e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.238238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb379f83-ad88-47e0-ae0f-f102d1a4a90e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.238509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb379f83-ad88-47e0-ae0f-f102d1a4a90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.385923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e827a14-807a-47c7-9127-28e015af44bd None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:07.388226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:07.389026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb379f83-ad88-47e0-ae0f-f102d1a4a90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:07.390499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb379f83-ad88-47e0-ae0f-f102d1a4a90e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:07.444506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415ee31-d57c-4e3a-a96c-962f2f4ddc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:07.445256 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415ee31-d57c-4e3a-a96c-962f2f4ddc40 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.445627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415ee31-d57c-4e3a-a96c-962f2f4ddc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.445904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415ee31-d57c-4e3a-a96c-962f2f4ddc40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.446620 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415ee31-d57c-4e3a-a96c-962f2f4ddc40 None admin] RBAC: Adding query filter params (role.id=0062155b0fbf41a19c9110ad71a046e3, scope.project.id=04853db2b388437e916da478e79b0816, user.id=6b6757539b0548b8a38441652f18bce9) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:07.446919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415ee31-d57c-4e3a-a96c-962f2f4ddc40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:07.448446 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415ee31-d57c-4e3a-a96c-962f2f4ddc40 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:07.448722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:07.459368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 79/158] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/role_assignments?user.id=6b6757539b0548b8a38441652f18bce9&scope.project.id=04853db2b388437e916da478e79b0816&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:07.459368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415ee31-d57c-4e3a-a96c-962f2f4ddc40 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.524243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 80/159] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:09:07.525104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb379f83-ad88-47e0-ae0f-f102d1a4a90e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.587412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415ee31-d57c-4e3a-a96c-962f2f4ddc40 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:07.633258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd766-bc97-4db0-97cf-639df47cc1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:07.634012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd766-bc97-4db0-97cf-639df47cc1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.634291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd766-bc97-4db0-97cf-639df47cc1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.634559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd766-bc97-4db0-97cf-639df47cc1f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.652310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bd766-bc97-4db0-97cf-639df47cc1f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:07.665884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb379f83-ad88-47e0-ae0f-f102d1a4a90e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-437bd766-bc97-4db0-97cf-639df47cc1f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:07.670204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:07.671118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:07.672022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 80/160] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:07.672661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd766-bc97-4db0-97cf-639df47cc1f8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.736821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:07.741185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017f489-dd7b-4113-9af9-3e760f2b9f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:07.743173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017f489-dd7b-4113-9af9-3e760f2b9f90 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.744265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017f489-dd7b-4113-9af9-3e760f2b9f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.744966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017f489-dd7b-4113-9af9-3e760f2b9f90 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.750532 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:07.772464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017f489-dd7b-4113-9af9-3e760f2b9f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-9017f489-dd7b-4113-9af9-3e760f2b9f90 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:07.778416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:07.779296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:07.780111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:07.787693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 81/161] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:07.789289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b97a-e832-40f6-bed0-27f371377086 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.789576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b97a-e832-40f6-bed0-27f371377086 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.791148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b97a-e832-40f6-bed0-27f371377086 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.791364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 82/162] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:07.791364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b97a-e832-40f6-bed0-27f371377086 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.840250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd766-bc97-4db0-97cf-639df47cc1f8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:07.877548 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206fd65-05c9-4de0-b5f4-5874052dc4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:07.878581 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206fd65-05c9-4de0-b5f4-5874052dc4bf None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:07.879092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206fd65-05c9-4de0-b5f4-5874052dc4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:07.879574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206fd65-05c9-4de0-b5f4-5874052dc4bf None admin] PATH_INFO: `/v3/projects/babc78c64eed4065a5c1558dc4043b4c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:07.881021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206fd65-05c9-4de0-b5f4-5874052dc4bf None admin] RBAC: Authorizing `identity:get_project(project_id=babc78c64eed4065a5c1558dc4043b4c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:07.882758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206fd65-05c9-4de0-b5f4-5874052dc4bf None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:07.884858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 81/163] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/projects/babc78c64eed4065a5c1558dc4043b4c => generated 312 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:07.886035 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206fd65-05c9-4de0-b5f4-5874052dc4bf None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:07.979311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b97a-e832-40f6-bed0-27f371377086 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:07.988671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:07.996347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206fd65-05c9-4de0-b5f4-5874052dc4bf None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.034100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae2db6-3b68-4f0c-b6a0-e1a31aff3cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.035057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae2db6-3b68-4f0c-b6a0-e1a31aff3cdc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.035526 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae2db6-3b68-4f0c-b6a0-e1a31aff3cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.036011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae2db6-3b68-4f0c-b6a0-e1a31aff3cdc None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.036864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae2db6-3b68-4f0c-b6a0-e1a31aff3cdc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:08.037145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfa78f-ae8d-4376-a4fb-2393cf7209d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.037677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfa78f-ae8d-4376-a4fb-2393cf7209d0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.037884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae2db6-3b68-4f0c-b6a0-e1a31aff3cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.037990 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfa78f-ae8d-4376-a4fb-2393cf7209d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.038218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfa78f-ae8d-4376-a4fb-2393cf7209d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.038880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfa78f-ae8d-4376-a4fb-2393cf7209d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:08.039182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfa78f-ae8d-4376-a4fb-2393cf7209d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.040255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae2db6-3b68-4f0c-b6a0-e1a31aff3cdc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.040598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfa78f-ae8d-4376-a4fb-2393cf7209d0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.050393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 82/164] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:08.052083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370be072-8370-4ab9-9233-a8ec0dcfc3a4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.053191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370be072-8370-4ab9-9233-a8ec0dcfc3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.053670 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370be072-8370-4ab9-9233-a8ec0dcfc3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.054022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 83/165] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:08.054863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfa78f-ae8d-4376-a4fb-2393cf7209d0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.074411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-370be072-8370-4ab9-9233-a8ec0dcfc3a4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:08.076090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-370be072-8370-4ab9-9233-a8ec0dcfc3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:08.151081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 83/166] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:08.155692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370be072-8370-4ab9-9233-a8ec0dcfc3a4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.174095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:08.279938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:08.284359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfa78f-ae8d-4376-a4fb-2393cf7209d0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.292752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370be072-8370-4ab9-9233-a8ec0dcfc3a4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.329083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70ace6-cfa8-4cac-aca4-5a426a7f7915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.329930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70ace6-cfa8-4cac-aca4-5a426a7f7915 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.330438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70ace6-cfa8-4cac-aca4-5a426a7f7915 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.330804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70ace6-cfa8-4cac-aca4-5a426a7f7915 None admin] PATH_INFO: `/v3/users/6b6757539b0548b8a38441652f18bce9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.332147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70ace6-cfa8-4cac-aca4-5a426a7f7915 None admin] RBAC: Authorizing `identity:get_user(user_id=6b6757539b0548b8a38441652f18bce9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.333703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70ace6-cfa8-4cac-aca4-5a426a7f7915 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.335660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 84/167] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/users/6b6757539b0548b8a38441652f18bce9 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:08.336591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70ace6-cfa8-4cac-aca4-5a426a7f7915 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.343712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d9e67-ffa0-4ad5-826a-34342b494b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.344341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d9e67-ffa0-4ad5-826a-34342b494b5b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.344616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d9e67-ffa0-4ad5-826a-34342b494b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.344885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d9e67-ffa0-4ad5-826a-34342b494b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.345615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d9e67-ffa0-4ad5-826a-34342b494b5b None admin] RBAC: Adding query filter params (role.id=0062155b0fbf41a19c9110ad71a046e3, scope.project.id=babc78c64eed4065a5c1558dc4043b4c, user.id=6b6757539b0548b8a38441652f18bce9) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:08.345922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d9e67-ffa0-4ad5-826a-34342b494b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.347378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d9e67-ffa0-4ad5-826a-34342b494b5b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.353753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 84/168] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/role_assignments?user.id=6b6757539b0548b8a38441652f18bce9&scope.project.id=babc78c64eed4065a5c1558dc4043b4c&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:08.354391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d9e67-ffa0-4ad5-826a-34342b494b5b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.449944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70ace6-cfa8-4cac-aca4-5a426a7f7915 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.489450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392205f8-e0ec-4878-b88f-a8c5a08d7cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.490349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392205f8-e0ec-4878-b88f-a8c5a08d7cfd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.490807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392205f8-e0ec-4878-b88f-a8c5a08d7cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.491178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392205f8-e0ec-4878-b88f-a8c5a08d7cfd None admin] PATH_INFO: `/v3/users/3015f5ce64524d2da234cc0cc671e2b5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.493554 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392205f8-e0ec-4878-b88f-a8c5a08d7cfd None admin] RBAC: Authorizing `identity:get_user(user_id=3015f5ce64524d2da234cc0cc671e2b5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.495378 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392205f8-e0ec-4878-b88f-a8c5a08d7cfd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.497215 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 85/169] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/users/3015f5ce64524d2da234cc0cc671e2b5 => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:08.503185 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392205f8-e0ec-4878-b88f-a8c5a08d7cfd None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.532400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d9e67-ffa0-4ad5-826a-34342b494b5b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.539899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:08.565594 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:08.596340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e9ea3-8c29-48fe-a156-4ccddae70d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.596984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e9ea3-8c29-48fe-a156-4ccddae70d44 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.597323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e9ea3-8c29-48fe-a156-4ccddae70d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.597604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e9ea3-8c29-48fe-a156-4ccddae70d44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.638738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392205f8-e0ec-4878-b88f-a8c5a08d7cfd None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.693372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc8bd8-0a97-4c78-a12b-0841c768d608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.694111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc8bd8-0a97-4c78-a12b-0841c768d608 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.694439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc8bd8-0a97-4c78-a12b-0841c768d608 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.694808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc8bd8-0a97-4c78-a12b-0841c768d608 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.697384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc8bd8-0a97-4c78-a12b-0841c768d608 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.699167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc8bd8-0a97-4c78-a12b-0841c768d608 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.702729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 86/170] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:08.705631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e9ea3-8c29-48fe-a156-4ccddae70d44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.707178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e9ea3-8c29-48fe-a156-4ccddae70d44 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.707710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc8bd8-0a97-4c78-a12b-0841c768d608 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.810719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 85/171] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:08.811927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dd36d-fcaa-46bd-a3ca-7c3fff01157a None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.812212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dd36d-fcaa-46bd-a3ca-7c3fff01157a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.812483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dd36d-fcaa-46bd-a3ca-7c3fff01157a None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.813727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 86/172] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:08.814422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dd36d-fcaa-46bd-a3ca-7c3fff01157a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:08.814940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:08.826221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc8bd8-0a97-4c78-a12b-0841c768d608 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.842041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:08.864450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcb28c-7e29-4399-9060-0244fac4a267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:08.865114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcb28c-7e29-4399-9060-0244fac4a267 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.865388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcb28c-7e29-4399-9060-0244fac4a267 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.865665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcb28c-7e29-4399-9060-0244fac4a267 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816/users/3015f5ce64524d2da234cc0cc671e2b5/roles/b19408717eaf492c9999c46062b83ba8` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.867750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcb28c-7e29-4399-9060-0244fac4a267 None admin] RBAC: Authorizing `identity:create_grant(project_id=04853db2b388437e916da478e79b0816, user_id=3015f5ce64524d2da234cc0cc671e2b5, role_id=b19408717eaf492c9999c46062b83ba8)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:08.869181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcb28c-7e29-4399-9060-0244fac4a267 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:08.932451 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 87/173] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:08 2022] PUT /identity/v3/projects/04853db2b388437e916da478e79b0816/users/3015f5ce64524d2da234cc0cc671e2b5/roles/b19408717eaf492c9999c46062b83ba8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:09:08.940635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b021511-2850-4fa3-a700-e8c7c6176143 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:08.940950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b021511-2850-4fa3-a700-e8c7c6176143 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:08.942254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b021511-2850-4fa3-a700-e8c7c6176143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:08.955681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dd36d-fcaa-46bd-a3ca-7c3fff01157a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:08.984956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1b021511-2850-4fa3-a700-e8c7c6176143 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:08.987061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1b021511-2850-4fa3-a700-e8c7c6176143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:09.033278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebb6bc-08b1-45c9-a3d5-8d0a312839f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:09.033929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebb6bc-08b1-45c9-a3d5-8d0a312839f4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.034328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebb6bc-08b1-45c9-a3d5-8d0a312839f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.034493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebb6bc-08b1-45c9-a3d5-8d0a312839f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.048518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebb6bc-08b1-45c9-a3d5-8d0a312839f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-a1ebb6bc-08b1-45c9-a3d5-8d0a312839f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:09.056042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:09.057030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 87/174] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:09.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebb6bc-08b1-45c9-a3d5-8d0a312839f4 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:09.124939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 88/175] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:09.125909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b021511-2850-4fa3-a700-e8c7c6176143 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:09.172370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:09.202406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebb6bc-08b1-45c9-a3d5-8d0a312839f4 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:09.243294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:09.259031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff626fe8-d71a-4f5a-a87d-2f3daddbf01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:09.260203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff626fe8-d71a-4f5a-a87d-2f3daddbf01d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.260995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff626fe8-d71a-4f5a-a87d-2f3daddbf01d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.261768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff626fe8-d71a-4f5a-a87d-2f3daddbf01d None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.263302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff626fe8-d71a-4f5a-a87d-2f3daddbf01d None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:09.265568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff626fe8-d71a-4f5a-a87d-2f3daddbf01d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:09.266420 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b021511-2850-4fa3-a700-e8c7c6176143 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:09.267740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 88/176] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:09 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:09.270091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb247d-ca88-4ddb-9ca4-d290d42f9688 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.270699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb247d-ca88-4ddb-9ca4-d290d42f9688 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.271173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb247d-ca88-4ddb-9ca4-d290d42f9688 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.272706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 89/177] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:09.273576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb247d-ca88-4ddb-9ca4-d290d42f9688 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:09.314166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f4019-3dda-461b-9f79-5f424d0e67dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:09.315289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f4019-3dda-461b-9f79-5f424d0e67dc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.315663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f4019-3dda-461b-9f79-5f424d0e67dc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.315998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f4019-3dda-461b-9f79-5f424d0e67dc None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.317011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f4019-3dda-461b-9f79-5f424d0e67dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:09.317379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f4019-3dda-461b-9f79-5f424d0e67dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:09.319491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f4019-3dda-461b-9f79-5f424d0e67dc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:09.329984 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 89/178] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:09.335562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b74011-13a3-4b84-8e38-89c27765fe3e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.335562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b74011-13a3-4b84-8e38-89c27765fe3e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.335562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b74011-13a3-4b84-8e38-89c27765fe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.358139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f1b74011-13a3-4b84-8e38-89c27765fe3e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:09.359457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f1b74011-13a3-4b84-8e38-89c27765fe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:09.436083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 90/179] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:09.437451 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b74011-13a3-4b84-8e38-89c27765fe3e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:09.509719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:09.544774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b74011-13a3-4b84-8e38-89c27765fe3e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:09.550736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb247d-ca88-4ddb-9ca4-d290d42f9688 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:09.587062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4ed12-c9f7-4282-902b-6eef0ceab7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:09.587748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4ed12-c9f7-4282-902b-6eef0ceab7cc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.588177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4ed12-c9f7-4282-902b-6eef0ceab7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.588446 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4ed12-c9f7-4282-902b-6eef0ceab7cc None admin] PATH_INFO: `/v3/users/57929111b78840da8ec77704273d56d2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.590213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4ed12-c9f7-4282-902b-6eef0ceab7cc None admin] RBAC: Authorizing `identity:get_user(user_id=57929111b78840da8ec77704273d56d2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:09.592417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4ed12-c9f7-4282-902b-6eef0ceab7cc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:09.594344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 91/180] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:09 2022] GET /identity/v3/users/57929111b78840da8ec77704273d56d2 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:09.595036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee033f-7c58-4c51-b305-dc96518af158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:09.595749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee033f-7c58-4c51-b305-dc96518af158 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.595855 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26f723-21ae-40cf-9bf5-45591b62f4ac None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.596030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26f723-21ae-40cf-9bf5-45591b62f4ac None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.596122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee033f-7c58-4c51-b305-dc96518af158 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.596280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26f723-21ae-40cf-9bf5-45591b62f4ac None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.596375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee033f-7c58-4c51-b305-dc96518af158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.597007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee033f-7c58-4c51-b305-dc96518af158 None admin] RBAC: Adding query filter params (role.id=43008618c1dc4cd49109f35c778080d3, scope.project.id=04853db2b388437e916da478e79b0816, user.id=6b6757539b0548b8a38441652f18bce9) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:09.597311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee033f-7c58-4c51-b305-dc96518af158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:09.597480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 92/181] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:09.598212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26f723-21ae-40cf-9bf5-45591b62f4ac None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:09.599004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee033f-7c58-4c51-b305-dc96518af158 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:09.605530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 90/182] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:09 2022] GET /identity/v3/role_assignments?user.id=6b6757539b0548b8a38441652f18bce9&scope.project.id=04853db2b388437e916da478e79b0816&role.id=43008618c1dc4cd49109f35c778080d3 => generated 608 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:09.606892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee033f-7c58-4c51-b305-dc96518af158 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:09.631349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:09.750537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26f723-21ae-40cf-9bf5-45591b62f4ac None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:09.767465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee033f-7c58-4c51-b305-dc96518af158 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:09.805602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ab7b9-8628-4a28-888b-2d23bd319568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:09.806409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ab7b9-8628-4a28-888b-2d23bd319568 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.806585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ab7b9-8628-4a28-888b-2d23bd319568 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.806844 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ab7b9-8628-4a28-888b-2d23bd319568 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.814375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5a1ce-8551-4a0a-82b2-c13fc7b37c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:09.814994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5a1ce-8551-4a0a-82b2-c13fc7b37c81 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.815270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5a1ce-8551-4a0a-82b2-c13fc7b37c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.815557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5a1ce-8551-4a0a-82b2-c13fc7b37c81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.816688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:09.819913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ab7b9-8628-4a28-888b-2d23bd319568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-0f2ab7b9-8628-4a28-888b-2d23bd319568 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:09.825436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:09.832551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 93/183] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:09.833351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b54b8e-6b51-4f8e-bde3-9f951b15bc50 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:09.834077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b54b8e-6b51-4f8e-bde3-9f951b15bc50 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:09.834077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b54b8e-6b51-4f8e-bde3-9f951b15bc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:09.851916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-88b54b8e-6b51-4f8e-bde3-9f951b15bc50 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:09.853403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-88b54b8e-6b51-4f8e-bde3-9f951b15bc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:09.906037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5a1ce-8551-4a0a-82b2-c13fc7b37c81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:09.907699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5a1ce-8551-4a0a-82b2-c13fc7b37c81 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:09.921554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:09.936301 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 94/184] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:09.943571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b54b8e-6b51-4f8e-bde3-9f951b15bc50 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:10.032328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 91/185] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:10.036445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5a1ce-8551-4a0a-82b2-c13fc7b37c81 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:10.055385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b54b8e-6b51-4f8e-bde3-9f951b15bc50 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:10.087433 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:10.094421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064813af-2be4-49d2-8f99-90487acc4e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:10.095054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064813af-2be4-49d2-8f99-90487acc4e8b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:10.095376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064813af-2be4-49d2-8f99-90487acc4e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:10.095683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064813af-2be4-49d2-8f99-90487acc4e8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:10.108152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064813af-2be4-49d2-8f99-90487acc4e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-064813af-2be4-49d2-8f99-90487acc4e8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:10.113776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:10.114644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:10.115439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 95/186] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:10.116123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064813af-2be4-49d2-8f99-90487acc4e8b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:10.155063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5a1ce-8551-4a0a-82b2-c13fc7b37c81 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:10.169671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:10.191476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224ce90-d088-4924-8b3d-acb0dcbe8f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:10.192482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224ce90-d088-4924-8b3d-acb0dcbe8f24 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:10.192952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224ce90-d088-4924-8b3d-acb0dcbe8f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:10.193524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224ce90-d088-4924-8b3d-acb0dcbe8f24 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:10.194375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224ce90-d088-4924-8b3d-acb0dcbe8f24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:10.194863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224ce90-d088-4924-8b3d-acb0dcbe8f24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:10.196465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224ce90-d088-4924-8b3d-acb0dcbe8f24 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:10.210025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 92/187] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:10.211236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224ce90-d088-4924-8b3d-acb0dcbe8f24 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:10.233205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064813af-2be4-49d2-8f99-90487acc4e8b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:10.267716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c162e-7c2d-4d07-afb9-dcfc5c1bf82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:10.268367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c162e-7c2d-4d07-afb9-dcfc5c1bf82f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:10.268646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c162e-7c2d-4d07-afb9-dcfc5c1bf82f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:10.268918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c162e-7c2d-4d07-afb9-dcfc5c1bf82f None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:10.270137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c162e-7c2d-4d07-afb9-dcfc5c1bf82f None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:10.271656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c162e-7c2d-4d07-afb9-dcfc5c1bf82f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:10.273395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 96/188] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:10 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:10.274204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c162e-7c2d-4d07-afb9-dcfc5c1bf82f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:10.326351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224ce90-d088-4924-8b3d-acb0dcbe8f24 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:10.335314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:10.367268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f7a6b-2d83-4583-baf8-897049e81b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:10.368556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f7a6b-2d83-4583-baf8-897049e81b69 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:10.368982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f7a6b-2d83-4583-baf8-897049e81b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:10.370094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f7a6b-2d83-4583-baf8-897049e81b69 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:10.371353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f7a6b-2d83-4583-baf8-897049e81b69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:10.372255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f7a6b-2d83-4583-baf8-897049e81b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:10.374326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f7a6b-2d83-4583-baf8-897049e81b69 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:10.377714 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c162e-7c2d-4d07-afb9-dcfc5c1bf82f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:10.386825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 93/189] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:10.387829 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f7a6b-2d83-4583-baf8-897049e81b69 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:10.417864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da5e7b-0fcf-44d7-bb0d-3c587b6e28ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:10.418494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5e7b-0fcf-44d7-bb0d-3c587b6e28ba None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:10.419323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5e7b-0fcf-44d7-bb0d-3c587b6e28ba None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:10.419621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5e7b-0fcf-44d7-bb0d-3c587b6e28ba None admin] PATH_INFO: `/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:10.429715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:10.433867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da5e7b-0fcf-44d7-bb0d-3c587b6e28ba None admin] RBAC: Authorizing `identity:get_user(user_id=b09585e90e3d40d2a2a3e9a1cdf34b08)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:10.435385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da5e7b-0fcf-44d7-bb0d-3c587b6e28ba None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:10.437048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 97/190] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:10 2022] GET /identity/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:10.441895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da5e7b-0fcf-44d7-bb0d-3c587b6e28ba None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:10.500997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f7a6b-2d83-4583-baf8-897049e81b69 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:10.536642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6931e8d-748e-4b1b-8db4-078deaa19bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:10.537512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6931e8d-748e-4b1b-8db4-078deaa19bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:10.537939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6931e8d-748e-4b1b-8db4-078deaa19bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:10.538212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6931e8d-748e-4b1b-8db4-078deaa19bb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:10.539317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6931e8d-748e-4b1b-8db4-078deaa19bb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:10.540807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6931e8d-748e-4b1b-8db4-078deaa19bb0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:10.542506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 94/191] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:10.545893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da5e7b-0fcf-44d7-bb0d-3c587b6e28ba None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:10.548134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6931e8d-748e-4b1b-8db4-078deaa19bb0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:10.563421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:10.582383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601adb07-d856-4a71-b631-92d4fac409f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:10.583376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601adb07-d856-4a71-b631-92d4fac409f3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:10.583376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601adb07-d856-4a71-b631-92d4fac409f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:10.583567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601adb07-d856-4a71-b631-92d4fac409f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:10.584195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601adb07-d856-4a71-b631-92d4fac409f3 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=04853db2b388437e916da478e79b0816, user.id=57929111b78840da8ec77704273d56d2) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:10.584496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601adb07-d856-4a71-b631-92d4fac409f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:10.586192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601adb07-d856-4a71-b631-92d4fac409f3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:10.592665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 98/192] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:10 2022] GET /identity/v3/role_assignments?user.id=57929111b78840da8ec77704273d56d2&scope.project.id=04853db2b388437e916da478e79b0816&role.id=705907b9bdec47cebf88b868a6a56356 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:10.604598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601adb07-d856-4a71-b631-92d4fac409f3 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:10.671311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6931e8d-748e-4b1b-8db4-078deaa19bb0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:10.674850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:10.711356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3b76f-2a54-4de1-849d-169a6a6a9bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:10.712044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3b76f-2a54-4de1-849d-169a6a6a9bae None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:10.712376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3b76f-2a54-4de1-849d-169a6a6a9bae None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:10.712649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3b76f-2a54-4de1-849d-169a6a6a9bae None admin] PATH_INFO: `/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:10.713819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3b76f-2a54-4de1-849d-169a6a6a9bae None admin] RBAC: Authorizing `identity:get_user(user_id=b09585e90e3d40d2a2a3e9a1cdf34b08)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:10.715371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3b76f-2a54-4de1-849d-169a6a6a9bae None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:10.718061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 95/193] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:10 2022] GET /identity/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:10.723618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3b76f-2a54-4de1-849d-169a6a6a9bae None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:10.743865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601adb07-d856-4a71-b631-92d4fac409f3 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:10.791401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6960c-4918-47f3-aba5-21d98f4f52ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:10.793802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6960c-4918-47f3-aba5-21d98f4f52ff None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:10.793802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6960c-4918-47f3-aba5-21d98f4f52ff None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:10.793802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6960c-4918-47f3-aba5-21d98f4f52ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:10.855809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3b76f-2a54-4de1-849d-169a6a6a9bae None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:10.861261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:10.880924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6960c-4918-47f3-aba5-21d98f4f52ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:10.882381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6960c-4918-47f3-aba5-21d98f4f52ff None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:10.892770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53d65d-2f64-400d-bc53-f9baa9b6be43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:10.893382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53d65d-2f64-400d-bc53-f9baa9b6be43 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:10.893659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53d65d-2f64-400d-bc53-f9baa9b6be43 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:10.893929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53d65d-2f64-400d-bc53-f9baa9b6be43 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:10.894622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53d65d-2f64-400d-bc53-f9baa9b6be43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:10.896038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53d65d-2f64-400d-bc53-f9baa9b6be43 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:10.983019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 99/194] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:10.983888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6960c-4918-47f3-aba5-21d98f4f52ff None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:11.007571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 96/195] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:09:10 2022] POST /identity/v3/users => generated 297 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:11.007571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53d65d-2f64-400d-bc53-f9baa9b6be43 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:11.056957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:11.101445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:11.125004 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6960c-4918-47f3-aba5-21d98f4f52ff None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:11.157780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53d65d-2f64-400d-bc53-f9baa9b6be43 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:11.161085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07cb31-7d7d-42d3-bc53-2819534cf7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:11.161697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07cb31-7d7d-42d3-bc53-2819534cf7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:11.161971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07cb31-7d7d-42d3-bc53-2819534cf7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:11.162239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07cb31-7d7d-42d3-bc53-2819534cf7e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:11.197869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6157d-1ece-4fab-abf1-a760e01670be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:11.198573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6157d-1ece-4fab-abf1-a760e01670be None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:11.198770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6157d-1ece-4fab-abf1-a760e01670be None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:11.199087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6157d-1ece-4fab-abf1-a760e01670be None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:11.201664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6157d-1ece-4fab-abf1-a760e01670be None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:11.203148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6157d-1ece-4fab-abf1-a760e01670be None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:11.206469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 97/196] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:11 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:11.206469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932c2e0-83a8-4a8e-bb33-a03059ace1b8 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:11.206469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932c2e0-83a8-4a8e-bb33-a03059ace1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:11.206723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932c2e0-83a8-4a8e-bb33-a03059ace1b8 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:11.209060 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 98/197] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:11.213670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932c2e0-83a8-4a8e-bb33-a03059ace1b8 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:11.251121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07cb31-7d7d-42d3-bc53-2819534cf7e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:11.252709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07cb31-7d7d-42d3-bc53-2819534cf7e0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:11.337765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:11.347476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932c2e0-83a8-4a8e-bb33-a03059ace1b8 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:11.355265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:11.379138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 100/198] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:11.383619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e5e82-a6de-4800-b0cd-889a6f8a18c6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:11.383619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e5e82-a6de-4800-b0cd-889a6f8a18c6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:11.383619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e5e82-a6de-4800-b0cd-889a6f8a18c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:11.393234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0a00a-a3db-414e-9082-20675cdccf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:11.394035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0a00a-a3db-414e-9082-20675cdccf00 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:11.394461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0a00a-a3db-414e-9082-20675cdccf00 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:11.395322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0a00a-a3db-414e-9082-20675cdccf00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:11.396264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0a00a-a3db-414e-9082-20675cdccf00 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7, user.id=b09585e90e3d40d2a2a3e9a1cdf34b08) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:11.397008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0a00a-a3db-414e-9082-20675cdccf00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:11.398648 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0a00a-a3db-414e-9082-20675cdccf00 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:11.400701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-687e5e82-a6de-4800-b0cd-889a6f8a18c6 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:11.401905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-687e5e82-a6de-4800-b0cd-889a6f8a18c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:11.406229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 99/199] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:11 2022] GET /identity/v3/role_assignments?user.id=b09585e90e3d40d2a2a3e9a1cdf34b08&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=705907b9bdec47cebf88b868a6a56356 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:11.407281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0a00a-a3db-414e-9082-20675cdccf00 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:11.500344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 101/200] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:11.536900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0a00a-a3db-414e-9082-20675cdccf00 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:11.576654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40033141-8c17-4539-b190-792fd87f27e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:11.577300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40033141-8c17-4539-b190-792fd87f27e8 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:11.577575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40033141-8c17-4539-b190-792fd87f27e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:11.577844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40033141-8c17-4539-b190-792fd87f27e8 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:11.579046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40033141-8c17-4539-b190-792fd87f27e8 None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:11.580586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40033141-8c17-4539-b190-792fd87f27e8 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:11.582254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 100/201] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:11 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:11.588626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e5e82-a6de-4800-b0cd-889a6f8a18c6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:11.701918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e5e82-a6de-4800-b0cd-889a6f8a18c6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:11.713688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:11.738792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cf2e4-9874-4ea1-bf72-7d78cccb9011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:11.739700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf2e4-9874-4ea1-bf72-7d78cccb9011 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:11.740124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf2e4-9874-4ea1-bf72-7d78cccb9011 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:11.740600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf2e4-9874-4ea1-bf72-7d78cccb9011 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:11.741543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cf2e4-9874-4ea1-bf72-7d78cccb9011 None admin] RBAC: Adding query filter params (role.id=43008618c1dc4cd49109f35c778080d3, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7, user.id=b09585e90e3d40d2a2a3e9a1cdf34b08) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:11.741994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cf2e4-9874-4ea1-bf72-7d78cccb9011 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:11.743613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cf2e4-9874-4ea1-bf72-7d78cccb9011 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:11.749661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 102/202] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:11 2022] GET /identity/v3/role_assignments?user.id=b09585e90e3d40d2a2a3e9a1cdf34b08&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=43008618c1dc4cd49109f35c778080d3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:11.827467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40033141-8c17-4539-b190-792fd87f27e8 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:11.860518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:11.938568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40033141-8c17-4539-b190-792fd87f27e8 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:11.976168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8be94-0caa-43f2-999f-89becb4f00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:11.976713 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8be94-0caa-43f2-999f-89becb4f00f6 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:11.976996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8be94-0caa-43f2-999f-89becb4f00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:11.977271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8be94-0caa-43f2-999f-89becb4f00f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:11.987946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8be94-0caa-43f2-999f-89becb4f00f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-6ac8be94-0caa-43f2-999f-89becb4f00f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:11.993057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:11.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:11.994855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:11.999647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 101/203] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:12.001946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cf2e4-9874-4ea1-bf72-7d78cccb9011 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:12.139448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cf2e4-9874-4ea1-bf72-7d78cccb9011 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:12.180595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df83c9-c4be-43b5-9bdd-f81a29511f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:12.181580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df83c9-c4be-43b5-9bdd-f81a29511f8a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:12.182067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df83c9-c4be-43b5-9bdd-f81a29511f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:12.182547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df83c9-c4be-43b5-9bdd-f81a29511f8a None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:12.183427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df83c9-c4be-43b5-9bdd-f81a29511f8a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:12.183897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df83c9-c4be-43b5-9bdd-f81a29511f8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:12.186002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df83c9-c4be-43b5-9bdd-f81a29511f8a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:12.194710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 103/204] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:12.201579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8be94-0caa-43f2-999f-89becb4f00f6 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:12.338772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:12.350711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8be94-0caa-43f2-999f-89becb4f00f6 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:12.396583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341e9f0-36f3-4387-ad2c-c6657fbc3aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:12.398507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341e9f0-36f3-4387-ad2c-c6657fbc3aae None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:12.398507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341e9f0-36f3-4387-ad2c-c6657fbc3aae None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:12.398507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341e9f0-36f3-4387-ad2c-c6657fbc3aae None admin] PATH_INFO: `/v3/users/3015f5ce64524d2da234cc0cc671e2b5` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:12.399066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341e9f0-36f3-4387-ad2c-c6657fbc3aae None admin] RBAC: Authorizing `identity:get_user(user_id=3015f5ce64524d2da234cc0cc671e2b5)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:12.400592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341e9f0-36f3-4387-ad2c-c6657fbc3aae None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:12.402174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 102/205] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:12 2022] GET /identity/v3/users/3015f5ce64524d2da234cc0cc671e2b5 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:12.408209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df83c9-c4be-43b5-9bdd-f81a29511f8a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:12.464590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:12.582129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df83c9-c4be-43b5-9bdd-f81a29511f8a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:12.616808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6082b9-8871-49fb-af0b-235519661711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:12.617379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6082b9-8871-49fb-af0b-235519661711 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:12.617641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6082b9-8871-49fb-af0b-235519661711 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:12.617916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6082b9-8871-49fb-af0b-235519661711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:12.687225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6082b9-8871-49fb-af0b-235519661711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:12.688711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6082b9-8871-49fb-af0b-235519661711 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:12.729436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:12.816067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 104/206] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:12.824585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341e9f0-36f3-4387-ad2c-c6657fbc3aae None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:12.965851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341e9f0-36f3-4387-ad2c-c6657fbc3aae None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:13.011641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4f6df-d1ec-4a8f-a141-30954843bc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:13.012329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4f6df-d1ec-4a8f-a141-30954843bc11 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:13.012647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4f6df-d1ec-4a8f-a141-30954843bc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:13.012919 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4f6df-d1ec-4a8f-a141-30954843bc11 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:13.017036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4f6df-d1ec-4a8f-a141-30954843bc11 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:13.018493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4f6df-d1ec-4a8f-a141-30954843bc11 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:13.020154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 103/207] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:12 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:13.026272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6082b9-8871-49fb-af0b-235519661711 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:13.208798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6082b9-8871-49fb-af0b-235519661711 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:13.254401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cf06c-ebca-4b00-a477-b39b6b03073e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:13.254957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cf06c-ebca-4b00-a477-b39b6b03073e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:13.255239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cf06c-ebca-4b00-a477-b39b6b03073e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:13.255517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cf06c-ebca-4b00-a477-b39b6b03073e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:13.256290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cf06c-ebca-4b00-a477-b39b6b03073e None admin] RBAC: Adding query filter params (role.id=b19408717eaf492c9999c46062b83ba8, scope.project.id=04853db2b388437e916da478e79b0816, user.id=3015f5ce64524d2da234cc0cc671e2b5) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:13.256596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cf06c-ebca-4b00-a477-b39b6b03073e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:13.258046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cf06c-ebca-4b00-a477-b39b6b03073e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:13.261337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:13.270840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 105/208] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:13 2022] GET /identity/v3/role_assignments?user.id=3015f5ce64524d2da234cc0cc671e2b5&scope.project.id=04853db2b388437e916da478e79b0816&role.id=b19408717eaf492c9999c46062b83ba8 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:13.402864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecb55c-0b0f-4a87-b992-b8b6ca726418 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:13.403834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecb55c-0b0f-4a87-b992-b8b6ca726418 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:13.404013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecb55c-0b0f-4a87-b992-b8b6ca726418 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:13.405639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 104/209] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:13.411048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78acf45-6144-4dfd-a22d-746e7a9e14cf None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:13.411366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78acf45-6144-4dfd-a22d-746e7a9e14cf None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:13.411628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78acf45-6144-4dfd-a22d-746e7a9e14cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:13.436549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e78acf45-6144-4dfd-a22d-746e7a9e14cf None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:13.437814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e78acf45-6144-4dfd-a22d-746e7a9e14cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:13.529800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31949710-3c23-4b96-8cde-98f8d6c80a81 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:13.530073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31949710-3c23-4b96-8cde-98f8d6c80a81 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:13.530630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 106/210] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:13.530989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31949710-3c23-4b96-8cde-98f8d6c80a81 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:13.536975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 105/211] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:13.542193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90211ed1-6392-4806-b5d5-8deff0774e47 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:13.542481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90211ed1-6392-4806-b5d5-8deff0774e47 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:13.542760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90211ed1-6392-4806-b5d5-8deff0774e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:13.564761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-90211ed1-6392-4806-b5d5-8deff0774e47 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:13.565979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-90211ed1-6392-4806-b5d5-8deff0774e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:13.638879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 107/212] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:13.848280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31949710-3c23-4b96-8cde-98f8d6c80a81 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:13.880197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:13.960973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31949710-3c23-4b96-8cde-98f8d6c80a81 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:13.975186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90211ed1-6392-4806-b5d5-8deff0774e47 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.000193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fae19-41a4-4ca4-8051-183c0597932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.000882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fae19-41a4-4ca4-8051-183c0597932b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.001102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fae19-41a4-4ca4-8051-183c0597932b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.001374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fae19-41a4-4ca4-8051-183c0597932b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.013341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fae19-41a4-4ca4-8051-183c0597932b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-b36fae19-41a4-4ca4-8051-183c0597932b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:14.018193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:14.019141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:14.020342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 106/213] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:14.021357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2dadb-7a7a-4e57-86ed-da25eb96ebf2 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.021357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2dadb-7a7a-4e57-86ed-da25eb96ebf2 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.021357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2dadb-7a7a-4e57-86ed-da25eb96ebf2 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.022528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 107/214] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:14.025572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2dadb-7a7a-4e57-86ed-da25eb96ebf2 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.052033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:14.094476 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90211ed1-6392-4806-b5d5-8deff0774e47 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.129756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:14.130933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ecc44-5a80-456d-913e-44d9b42ceb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.131959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ecc44-5a80-456d-913e-44d9b42ceb47 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.132376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ecc44-5a80-456d-913e-44d9b42ceb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.132491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ecc44-5a80-456d-913e-44d9b42ceb47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.133609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ecc44-5a80-456d-913e-44d9b42ceb47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:14.135186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ecc44-5a80-456d-913e-44d9b42ceb47 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:14.136715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 108/215] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:14.142588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa11b4-0c22-4ccb-abca-fc91d9fd201c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.142588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa11b4-0c22-4ccb-abca-fc91d9fd201c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.142588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa11b4-0c22-4ccb-abca-fc91d9fd201c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.151644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2dadb-7a7a-4e57-86ed-da25eb96ebf2 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.156672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9daa11b4-0c22-4ccb-abca-fc91d9fd201c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:14.157859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9daa11b4-0c22-4ccb-abca-fc91d9fd201c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:14.187161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3709c-f8bd-414a-9578-8a84355d7a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.187799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3709c-f8bd-414a-9578-8a84355d7a44 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.188082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3709c-f8bd-414a-9578-8a84355d7a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.188363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3709c-f8bd-414a-9578-8a84355d7a44 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.189018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3709c-f8bd-414a-9578-8a84355d7a44 None admin] RBAC: Adding query filter params (name=member) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:14.189320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3709c-f8bd-414a-9578-8a84355d7a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:14.190702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3709c-f8bd-414a-9578-8a84355d7a44 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:14.203930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 108/216] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:14.203930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3709c-f8bd-414a-9578-8a84355d7a44 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.239829 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 109/217] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:14.240715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa11b4-0c22-4ccb-abca-fc91d9fd201c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.302178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:14.306887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3709c-f8bd-414a-9578-8a84355d7a44 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.341436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc7e20-31e5-449e-b02e-23afc2f772a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.342042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc7e20-31e5-449e-b02e-23afc2f772a8 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.342323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc7e20-31e5-449e-b02e-23afc2f772a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.342596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc7e20-31e5-449e-b02e-23afc2f772a8 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.343237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc7e20-31e5-449e-b02e-23afc2f772a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:14.344717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc7e20-31e5-449e-b02e-23afc2f772a8 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:14.346469 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa11b4-0c22-4ccb-abca-fc91d9fd201c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.381017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28234de4-865c-4a49-8253-3046c341c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.381511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28234de4-865c-4a49-8253-3046c341c219 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.381787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28234de4-865c-4a49-8253-3046c341c219 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.382059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28234de4-865c-4a49-8253-3046c341c219 None admin] PATH_INFO: `/v3/users/be7a1a21ccad4a6595a504abd0e43ce5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.395586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28234de4-865c-4a49-8253-3046c341c219 None admin] RBAC: Authorizing `identity:get_user(user_id=be7a1a21ccad4a6595a504abd0e43ce5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:14.397029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28234de4-865c-4a49-8253-3046c341c219 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:14.398688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 110/218] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:14 2022] GET /identity/v3/users/be7a1a21ccad4a6595a504abd0e43ce5 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:14.399841 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d14d23-4b6b-41dd-a237-486786ddc191 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.402708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d14d23-4b6b-41dd-a237-486786ddc191 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.402708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d14d23-4b6b-41dd-a237-486786ddc191 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.402708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 111/219] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:14.405432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d14d23-4b6b-41dd-a237-486786ddc191 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.456325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 109/220] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:09:14 2022] POST /identity/v3/users => generated 297 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:14.469189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2e32f-b822-450b-a390-a2290abd8e0e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.469556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2e32f-b822-450b-a390-a2290abd8e0e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.469942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2e32f-b822-450b-a390-a2290abd8e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.478162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:14.496670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-4aa2e32f-b822-450b-a390-a2290abd8e0e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:14.498502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4aa2e32f-b822-450b-a390-a2290abd8e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:14.524824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d14d23-4b6b-41dd-a237-486786ddc191 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.544658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:14.564548 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aa5fd-de7b-4065-a734-5b911df09115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.565187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aa5fd-de7b-4065-a734-5b911df09115 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.565464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aa5fd-de7b-4065-a734-5b911df09115 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.565736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aa5fd-de7b-4065-a734-5b911df09115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.584117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 110/221] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:14.585248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2e32f-b822-450b-a390-a2290abd8e0e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.662421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23aa5fd-de7b-4065-a734-5b911df09115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:14.664012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23aa5fd-de7b-4065-a734-5b911df09115 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:14.723720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2e32f-b822-450b-a390-a2290abd8e0e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.772953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:14.782248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4730f-44ca-4c5e-915b-92912edca176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.782899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4730f-44ca-4c5e-915b-92912edca176 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.789215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4730f-44ca-4c5e-915b-92912edca176 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.789215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4730f-44ca-4c5e-915b-92912edca176 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.796837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4730f-44ca-4c5e-915b-92912edca176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:14.799084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 112/222] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-2ba4730f-44ca-4c5e-915b-92912edca176 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:14.802169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:14.803029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:14.803855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 111/223] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:14.807080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aa5fd-de7b-4065-a734-5b911df09115 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.810692 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4730f-44ca-4c5e-915b-92912edca176 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.831949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:14.925261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aa5fd-de7b-4065-a734-5b911df09115 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.946212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4730f-44ca-4c5e-915b-92912edca176 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:14.967495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c102b-d2d4-48dc-b050-0f922e23e93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.968166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c102b-d2d4-48dc-b050-0f922e23e93f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.968446 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c102b-d2d4-48dc-b050-0f922e23e93f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.968720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c102b-d2d4-48dc-b050-0f922e23e93f None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.969932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c102b-d2d4-48dc-b050-0f922e23e93f None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:14.972152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c102b-d2d4-48dc-b050-0f922e23e93f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:14.974702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 113/224] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:14 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:14.975763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c102b-d2d4-48dc-b050-0f922e23e93f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:14.997727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fe63b-223c-41b3-a0b7-f554fd5faf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:14.998602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe63b-223c-41b3-a0b7-f554fd5faf2a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:14.998883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe63b-223c-41b3-a0b7-f554fd5faf2a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:14.999156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe63b-223c-41b3-a0b7-f554fd5faf2a None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:14.999876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fe63b-223c-41b3-a0b7-f554fd5faf2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:15.000181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fe63b-223c-41b3-a0b7-f554fd5faf2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:15.002293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fe63b-223c-41b3-a0b7-f554fd5faf2a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:15.015383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 112/225] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:15.020463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fe63b-223c-41b3-a0b7-f554fd5faf2a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:15.077568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:15.081414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:15.094673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c102b-d2d4-48dc-b050-0f922e23e93f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:15.129505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fe63b-223c-41b3-a0b7-f554fd5faf2a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:15.131495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b5531-4f41-4d01-8182-f5261b600e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:15.132364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5531-4f41-4d01-8182-f5261b600e35 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:15.132835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5531-4f41-4d01-8182-f5261b600e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:15.133300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5531-4f41-4d01-8182-f5261b600e35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:15.145176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b5531-4f41-4d01-8182-f5261b600e35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-859b5531-4f41-4d01-8182-f5261b600e35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:15.150333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:15.151277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:15.152093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:15.154461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 114/226] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:15.155529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b5531-4f41-4d01-8182-f5261b600e35 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:15.166820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465929b-0c27-45a6-ae32-494d504439c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:15.167500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465929b-0c27-45a6-ae32-494d504439c5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:15.168008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465929b-0c27-45a6-ae32-494d504439c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:15.168126 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465929b-0c27-45a6-ae32-494d504439c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:15.168880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465929b-0c27-45a6-ae32-494d504439c5 None admin] RBAC: Adding query filter params (role.id=0062155b0fbf41a19c9110ad71a046e3, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7, user.id=be7a1a21ccad4a6595a504abd0e43ce5) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:15.169192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465929b-0c27-45a6-ae32-494d504439c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:15.170660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465929b-0c27-45a6-ae32-494d504439c5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:15.176850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 113/227] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:15 2022] GET /identity/v3/role_assignments?user.id=be7a1a21ccad4a6595a504abd0e43ce5&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:15.177699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465929b-0c27-45a6-ae32-494d504439c5 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:15.268525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b5531-4f41-4d01-8182-f5261b600e35 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:15.294395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465929b-0c27-45a6-ae32-494d504439c5 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:15.308134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a96d1-56d2-4f19-abf6-fa21d5cd1688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:15.309227 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a96d1-56d2-4f19-abf6-fa21d5cd1688 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:15.309832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a96d1-56d2-4f19-abf6-fa21d5cd1688 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:15.310295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a96d1-56d2-4f19-abf6-fa21d5cd1688 None admin] PATH_INFO: `/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:15.311653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a96d1-56d2-4f19-abf6-fa21d5cd1688 None admin] RBAC: Authorizing `identity:get_user(user_id=b09585e90e3d40d2a2a3e9a1cdf34b08)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:15.313331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:15.314381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a96d1-56d2-4f19-abf6-fa21d5cd1688 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:15.316273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 115/228] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:15 2022] GET /identity/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:15.328607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a96d1-56d2-4f19-abf6-fa21d5cd1688 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:15.338547 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf3fc7-f0bc-4d52-ba57-8650827f19b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:15.339224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf3fc7-f0bc-4d52-ba57-8650827f19b3 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:15.339506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf3fc7-f0bc-4d52-ba57-8650827f19b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:15.341228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf3fc7-f0bc-4d52-ba57-8650827f19b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:15.341228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf3fc7-f0bc-4d52-ba57-8650827f19b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:15.341412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf3fc7-f0bc-4d52-ba57-8650827f19b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:15.342821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf3fc7-f0bc-4d52-ba57-8650827f19b3 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:15.358266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 114/229] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:15.358266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf3fc7-f0bc-4d52-ba57-8650827f19b3 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:15.362599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:15.516522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a96d1-56d2-4f19-abf6-fa21d5cd1688 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:15.565261 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf3fc7-f0bc-4d52-ba57-8650827f19b3 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:15.579117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de176a1a-81fe-4bd8-a8a2-61a3b00c3c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:15.579801 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de176a1a-81fe-4bd8-a8a2-61a3b00c3c92 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:15.580081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de176a1a-81fe-4bd8-a8a2-61a3b00c3c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:15.580353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de176a1a-81fe-4bd8-a8a2-61a3b00c3c92 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:15.583184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de176a1a-81fe-4bd8-a8a2-61a3b00c3c92 None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:15.584656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de176a1a-81fe-4bd8-a8a2-61a3b00c3c92 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:15.586579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 116/230] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:15 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:15.595132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de176a1a-81fe-4bd8-a8a2-61a3b00c3c92 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:15.634031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e968-07a5-42a0-b4ea-753223a41ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:15.634676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e968-07a5-42a0-b4ea-753223a41ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:15.634959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e968-07a5-42a0-b4ea-753223a41ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:15.635238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e968-07a5-42a0-b4ea-753223a41ca3 None admin] PATH_INFO: `/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:15.637372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781e968-07a5-42a0-b4ea-753223a41ca3 None admin] RBAC: Authorizing `identity:get_user(user_id=b09585e90e3d40d2a2a3e9a1cdf34b08)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:15.639464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781e968-07a5-42a0-b4ea-753223a41ca3 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:15.641084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 115/231] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:15 2022] GET /identity/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08 => generated 283 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:15.652409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e968-07a5-42a0-b4ea-753223a41ca3 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:15.745426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:15.746308 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:15.774345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de176a1a-81fe-4bd8-a8a2-61a3b00c3c92 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:15.836063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e968-07a5-42a0-b4ea-753223a41ca3 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:15.842722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d9b1c-ffd6-4977-8d9c-ac786ad0e3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:15.843772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d9b1c-ffd6-4977-8d9c-ac786ad0e3b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:15.844268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d9b1c-ffd6-4977-8d9c-ac786ad0e3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:15.844728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d9b1c-ffd6-4977-8d9c-ac786ad0e3b9 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/users/b09585e90e3d40d2a2a3e9a1cdf34b08/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:15.852329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d9b1c-ffd6-4977-8d9c-ac786ad0e3b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=88bda6b61b9c4485a3366fe286e2c1d7, user_id=b09585e90e3d40d2a2a3e9a1cdf34b08, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:15.854050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d9b1c-ffd6-4977-8d9c-ac786ad0e3b9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:15.873049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be56aa-df86-43df-9b21-492a738ebf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:15.873695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be56aa-df86-43df-9b21-492a738ebf5d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:15.873972 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be56aa-df86-43df-9b21-492a738ebf5d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:15.874243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be56aa-df86-43df-9b21-492a738ebf5d None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:15.875385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be56aa-df86-43df-9b21-492a738ebf5d None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:15.876925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be56aa-df86-43df-9b21-492a738ebf5d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:15.878989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 116/232] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:15 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:15.884598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be56aa-df86-43df-9b21-492a738ebf5d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:15.954214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 117/233] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:15 2022] PUT /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/users/b09585e90e3d40d2a2a3e9a1cdf34b08/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:16.013489 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be56aa-df86-43df-9b21-492a738ebf5d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:16.036032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:16.060835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedbb6b-e356-44e9-9c6e-e87d39e3de0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:16.061864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedbb6b-e356-44e9-9c6e-e87d39e3de0f None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:16.062358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedbb6b-e356-44e9-9c6e-e87d39e3de0f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:16.062706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedbb6b-e356-44e9-9c6e-e87d39e3de0f None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/users/b09585e90e3d40d2a2a3e9a1cdf34b08/roles/43008618c1dc4cd49109f35c778080d3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:16.068337 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedbb6b-e356-44e9-9c6e-e87d39e3de0f None admin] RBAC: Authorizing `identity:create_grant(project_id=88bda6b61b9c4485a3366fe286e2c1d7, user_id=b09585e90e3d40d2a2a3e9a1cdf34b08, role_id=43008618c1dc4cd49109f35c778080d3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:16.069896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedbb6b-e356-44e9-9c6e-e87d39e3de0f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:16.165363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 117/234] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:15 2022] PUT /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/users/b09585e90e3d40d2a2a3e9a1cdf34b08/roles/43008618c1dc4cd49109f35c778080d3 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:16.913299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443deaa-5e70-4a35-9f6a-5dc1df3682b2 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:16.913811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443deaa-5e70-4a35-9f6a-5dc1df3682b2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:16.913811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443deaa-5e70-4a35-9f6a-5dc1df3682b2 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:16.915169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 118/235] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:16.920985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9383bc4-6062-435f-941e-6c1da1bb87f0 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:16.921264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9383bc4-6062-435f-941e-6c1da1bb87f0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:16.921541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9383bc4-6062-435f-941e-6c1da1bb87f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:16.928107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d12d5-0157-4b79-b287-6a77294a9a90 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:16.928384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d12d5-0157-4b79-b287-6a77294a9a90 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:16.928653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d12d5-0157-4b79-b287-6a77294a9a90 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:16.930071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 119/236] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:09:16.936503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caaddc-5f4c-4bfe-aa47-84c4c2362f07 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:16.936776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caaddc-5f4c-4bfe-aa47-84c4c2362f07 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:16.937043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caaddc-5f4c-4bfe-aa47-84c4c2362f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:16.958755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e9383bc4-6062-435f-941e-6c1da1bb87f0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:16.959701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e9383bc4-6062-435f-941e-6c1da1bb87f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:16.968557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e0caaddc-5f4c-4bfe-aa47-84c4c2362f07 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:16.969726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e0caaddc-5f4c-4bfe-aa47-84c4c2362f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:17.031437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 118/237] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:17.109774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 120/238] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:09:17.341884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9383bc4-6062-435f-941e-6c1da1bb87f0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:17.475298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9383bc4-6062-435f-941e-6c1da1bb87f0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:17.486954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:17.501463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0caaddc-5f4c-4bfe-aa47-84c4c2362f07 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:17.532347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452ba53-3352-46aa-bb40-b096af51f80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:17.533002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452ba53-3352-46aa-bb40-b096af51f80f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.533278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452ba53-3352-46aa-bb40-b096af51f80f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.533552 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452ba53-3352-46aa-bb40-b096af51f80f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:17.545664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452ba53-3352-46aa-bb40-b096af51f80f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-e452ba53-3352-46aa-bb40-b096af51f80f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:17.550824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:17.552050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:17.552882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 119/239] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:17.553567 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8ed7c-a680-4388-bf1a-4639eef82b86 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.553567 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8ed7c-a680-4388-bf1a-4639eef82b86 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.554065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8ed7c-a680-4388-bf1a-4639eef82b86 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:17.555374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 120/240] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:17.561862 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8ed7c-a680-4388-bf1a-4639eef82b86 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:17.588438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:17.625886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0caaddc-5f4c-4bfe-aa47-84c4c2362f07 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:17.673210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8ed7c-a680-4388-bf1a-4639eef82b86 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:17.679184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143846e-6d35-4a66-bd9d-0643f8ee4f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:17.679971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143846e-6d35-4a66-bd9d-0643f8ee4f42 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.680320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143846e-6d35-4a66-bd9d-0643f8ee4f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.680661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143846e-6d35-4a66-bd9d-0643f8ee4f42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:17.681923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143846e-6d35-4a66-bd9d-0643f8ee4f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:17.683489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143846e-6d35-4a66-bd9d-0643f8ee4f42 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:17.685881 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 121/241] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:17.690654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b460d91-44a9-47df-a07c-efd39c620896 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.690654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b460d91-44a9-47df-a07c-efd39c620896 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.690654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b460d91-44a9-47df-a07c-efd39c620896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:17.706929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3b460d91-44a9-47df-a07c-efd39c620896 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:17.708267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3b460d91-44a9-47df-a07c-efd39c620896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:17.725272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a0c81-4104-4447-a399-74732dc2ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:17.726380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a0c81-4104-4447-a399-74732dc2ee89 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.726832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a0c81-4104-4447-a399-74732dc2ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.727659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a0c81-4104-4447-a399-74732dc2ee89 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:17.728511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a0c81-4104-4447-a399-74732dc2ee89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:17.728983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a0c81-4104-4447-a399-74732dc2ee89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:17.731080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a0c81-4104-4447-a399-74732dc2ee89 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:17.744962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 121/242] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:17.746093 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a0c81-4104-4447-a399-74732dc2ee89 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:17.784752 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:17.787724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 122/243] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:17.788699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b460d91-44a9-47df-a07c-efd39c620896 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:17.873166 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a0c81-4104-4447-a399-74732dc2ee89 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:17.914201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be523a7-f010-4047-8a37-9fc4a822e1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:17.915180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be523a7-f010-4047-8a37-9fc4a822e1c5 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:17.915902 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be523a7-f010-4047-8a37-9fc4a822e1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:17.916386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be523a7-f010-4047-8a37-9fc4a822e1c5 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:17.917251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be523a7-f010-4047-8a37-9fc4a822e1c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:17.919506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be523a7-f010-4047-8a37-9fc4a822e1c5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:17.928413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:17.972903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b460d91-44a9-47df-a07c-efd39c620896 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:18.010459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a788d-b5c7-4fe5-844d-b74decc8d0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:18.011502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a788d-b5c7-4fe5-844d-b74decc8d0ae None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.012022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a788d-b5c7-4fe5-844d-b74decc8d0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.012549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a788d-b5c7-4fe5-844d-b74decc8d0ae None admin] PATH_INFO: `/v3/users/a8aa6abe8fb14b40b437943e2a36c3e5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.032999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 122/244] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:09:17 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:18.039451 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a788d-b5c7-4fe5-844d-b74decc8d0ae None admin] RBAC: Authorizing `identity:get_user(user_id=a8aa6abe8fb14b40b437943e2a36c3e5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:18.041194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a788d-b5c7-4fe5-844d-b74decc8d0ae None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:18.043476 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 123/245] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:17 2022] GET /identity/v3/users/a8aa6abe8fb14b40b437943e2a36c3e5 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:18.057235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be523a7-f010-4047-8a37-9fc4a822e1c5 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.159137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a788d-b5c7-4fe5-844d-b74decc8d0ae None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.177817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:18.198326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be523a7-f010-4047-8a37-9fc4a822e1c5 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:18.251342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fbb7e-013f-4405-8edf-91b1febfb0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:18.252006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fbb7e-013f-4405-8edf-91b1febfb0fc None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.252287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fbb7e-013f-4405-8edf-91b1febfb0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.252628 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fbb7e-013f-4405-8edf-91b1febfb0fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.289590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a788d-b5c7-4fe5-844d-b74decc8d0ae None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:18.318728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:18.328558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f33d6-45a1-40cc-9fe7-87a4064ec796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:18.328558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f33d6-45a1-40cc-9fe7-87a4064ec796 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.328809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f33d6-45a1-40cc-9fe7-87a4064ec796 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.329036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f33d6-45a1-40cc-9fe7-87a4064ec796 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.334404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fbb7e-013f-4405-8edf-91b1febfb0fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:18.336314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fbb7e-013f-4405-8edf-91b1febfb0fc None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:18.349571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f33d6-45a1-40cc-9fe7-87a4064ec796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f63f33d6-45a1-40cc-9fe7-87a4064ec796 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:18.353459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:18.354346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:18.355223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 124/246] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:18.356918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b072fa-e94a-45a2-b56a-eb84de0cd244 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.359911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b072fa-e94a-45a2-b56a-eb84de0cd244 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.359911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b072fa-e94a-45a2-b56a-eb84de0cd244 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.359911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 125/247] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:18.363606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b072fa-e94a-45a2-b56a-eb84de0cd244 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.441662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:18.466812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 123/248] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:18.467012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a52eb-2396-4da4-8845-503d45117a3b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.467764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a52eb-2396-4da4-8845-503d45117a3b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.467764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a52eb-2396-4da4-8845-503d45117a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.490159 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7a4a52eb-2396-4da4-8845-503d45117a3b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:18.491172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b072fa-e94a-45a2-b56a-eb84de0cd244 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:18.491512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7a4a52eb-2396-4da4-8845-503d45117a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:18.534063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d40819-95b6-4310-845b-afb6fad6db7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:18.534678 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d40819-95b6-4310-845b-afb6fad6db7c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.534954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d40819-95b6-4310-845b-afb6fad6db7c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.535226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d40819-95b6-4310-845b-afb6fad6db7c None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.535922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d40819-95b6-4310-845b-afb6fad6db7c None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:18.536219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d40819-95b6-4310-845b-afb6fad6db7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:18.537577 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d40819-95b6-4310-845b-afb6fad6db7c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:18.547300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 126/249] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:18.547994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d40819-95b6-4310-845b-afb6fad6db7c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.583410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 124/250] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:18.584708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a52eb-2396-4da4-8845-503d45117a3b None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.603079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:18.659634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d40819-95b6-4310-845b-afb6fad6db7c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:18.701979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e9650-444d-4fbc-aa75-6fe3c0683424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:18.702585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e9650-444d-4fbc-aa75-6fe3c0683424 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.702865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e9650-444d-4fbc-aa75-6fe3c0683424 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.703135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e9650-444d-4fbc-aa75-6fe3c0683424 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.704340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e9650-444d-4fbc-aa75-6fe3c0683424 None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:18.705849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e9650-444d-4fbc-aa75-6fe3c0683424 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:18.707696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 127/251] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:18.708667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15970298-5098-480d-8c81-98c51f25a357 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.708941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15970298-5098-480d-8c81-98c51f25a357 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.709209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15970298-5098-480d-8c81-98c51f25a357 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.710852 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 128/252] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:18.715105 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15970298-5098-480d-8c81-98c51f25a357 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.721768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a52eb-2396-4da4-8845-503d45117a3b None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:18.741617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:18.771359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b0081-2ea7-4412-9a06-402a4c47d54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:18.772700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b0081-2ea7-4412-9a06-402a4c47d54b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.773203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b0081-2ea7-4412-9a06-402a4c47d54b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.773947 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b0081-2ea7-4412-9a06-402a4c47d54b None admin] PATH_INFO: `/v3/users/be7a1a21ccad4a6595a504abd0e43ce5` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.776140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b0081-2ea7-4412-9a06-402a4c47d54b None admin] RBAC: Authorizing `identity:get_user(user_id=be7a1a21ccad4a6595a504abd0e43ce5)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:18.777846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b0081-2ea7-4412-9a06-402a4c47d54b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:18.781569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 125/253] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/users/be7a1a21ccad4a6595a504abd0e43ce5 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:18.783706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd1b04-9cdf-48bd-8d4a-b6feb44d8fd4 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:18.784211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd1b04-9cdf-48bd-8d4a-b6feb44d8fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:18.784680 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd1b04-9cdf-48bd-8d4a-b6feb44d8fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:18.806336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6bbd1b04-9cdf-48bd-8d4a-b6feb44d8fd4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:18.807879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6bbd1b04-9cdf-48bd-8d4a-b6feb44d8fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:18.885213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 126/254] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:18.886060 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd1b04-9cdf-48bd-8d4a-b6feb44d8fd4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:18.924248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15970298-5098-480d-8c81-98c51f25a357 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:18.961689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:19.003289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd1b04-9cdf-48bd-8d4a-b6feb44d8fd4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.012413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43112c7c-14ad-498b-89b4-6fb6ab989e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.013392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43112c7c-14ad-498b-89b4-6fb6ab989e02 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.013668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43112c7c-14ad-498b-89b4-6fb6ab989e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.013936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43112c7c-14ad-498b-89b4-6fb6ab989e02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.014667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43112c7c-14ad-498b-89b4-6fb6ab989e02 None admin] RBAC: Adding query filter params (role.id=b19408717eaf492c9999c46062b83ba8, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7, user.id=a8aa6abe8fb14b40b437943e2a36c3e5) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:19.014974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43112c7c-14ad-498b-89b4-6fb6ab989e02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:19.016434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43112c7c-14ad-498b-89b4-6fb6ab989e02 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:19.020449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:19.033742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 129/255] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/role_assignments?user.id=a8aa6abe8fb14b40b437943e2a36c3e5&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=b19408717eaf492c9999c46062b83ba8 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:19.037902 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43112c7c-14ad-498b-89b4-6fb6ab989e02 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:19.045275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc2ebc-80c0-4161-b06c-c3cc118af9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.046542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc2ebc-80c0-4161-b06c-c3cc118af9cc None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.046965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc2ebc-80c0-4161-b06c-c3cc118af9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.047371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc2ebc-80c0-4161-b06c-c3cc118af9cc None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.050264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc2ebc-80c0-4161-b06c-c3cc118af9cc None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:19.052029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc2ebc-80c0-4161-b06c-c3cc118af9cc None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:19.054179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 127/256] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:18 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:19.060235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc2ebc-80c0-4161-b06c-c3cc118af9cc None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:19.252872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc2ebc-80c0-4161-b06c-c3cc118af9cc None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.287835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43112c7c-14ad-498b-89b4-6fb6ab989e02 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.328522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb2d71-e45d-4853-88cf-d2ece593ede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.329191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb2d71-e45d-4853-88cf-d2ece593ede7 None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.329467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb2d71-e45d-4853-88cf-d2ece593ede7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.329740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb2d71-e45d-4853-88cf-d2ece593ede7 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/users/be7a1a21ccad4a6595a504abd0e43ce5/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.334399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb2d71-e45d-4853-88cf-d2ece593ede7 None admin] RBAC: Authorizing `identity:create_grant(project_id=88bda6b61b9c4485a3366fe286e2c1d7, user_id=be7a1a21ccad4a6595a504abd0e43ce5, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:19.336143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb2d71-e45d-4853-88cf-d2ece593ede7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:19.353749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a816e-f949-4fbc-8042-9aa0feb98e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.354378 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a816e-f949-4fbc-8042-9aa0feb98e60 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.354655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a816e-f949-4fbc-8042-9aa0feb98e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.354926 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a816e-f949-4fbc-8042-9aa0feb98e60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.365484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a816e-f949-4fbc-8042-9aa0feb98e60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-ff1a816e-f949-4fbc-8042-9aa0feb98e60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:19.370882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:19.371802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:19.373163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 130/257] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:19.373857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a816e-f949-4fbc-8042-9aa0feb98e60 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:19.416314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 128/258] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:19 2022] PUT /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/users/be7a1a21ccad4a6595a504abd0e43ce5/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:19.417152 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb2d71-e45d-4853-88cf-d2ece593ede7 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:19.444129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:19.499694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:19.560538 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a816e-f949-4fbc-8042-9aa0feb98e60 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.611143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb2d71-e45d-4853-88cf-d2ece593ede7 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.628971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cef5da-28dc-4fd3-acbb-5245b9f1c957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.629624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef5da-28dc-4fd3-acbb-5245b9f1c957 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.629898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef5da-28dc-4fd3-acbb-5245b9f1c957 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.630169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef5da-28dc-4fd3-acbb-5245b9f1c957 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.661219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cef5da-28dc-4fd3-acbb-5245b9f1c957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-31cef5da-28dc-4fd3-acbb-5245b9f1c957 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:19.666634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:19.667490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:19.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:19.671869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 131/259] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:19.677586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cef5da-28dc-4fd3-acbb-5245b9f1c957 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:19.692614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64073b52-4dd5-403d-a5b0-08796da3ae5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.693239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64073b52-4dd5-403d-a5b0-08796da3ae5f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.693517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64073b52-4dd5-403d-a5b0-08796da3ae5f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.693784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64073b52-4dd5-403d-a5b0-08796da3ae5f None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.694460 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64073b52-4dd5-403d-a5b0-08796da3ae5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:19.694760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64073b52-4dd5-403d-a5b0-08796da3ae5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:19.696248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64073b52-4dd5-403d-a5b0-08796da3ae5f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:19.710632 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 129/260] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:19.724006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64073b52-4dd5-403d-a5b0-08796da3ae5f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:19.885359 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cef5da-28dc-4fd3-acbb-5245b9f1c957 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.887861 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:19.921901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64073b52-4dd5-403d-a5b0-08796da3ae5f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:19.946271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:19.986154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f757e-d128-4856-b3ea-4a555c2c1727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.987654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f757e-d128-4856-b3ea-4a555c2c1727 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.988137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f757e-d128-4856-b3ea-4a555c2c1727 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.988606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f757e-d128-4856-b3ea-4a555c2c1727 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.989492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f757e-d128-4856-b3ea-4a555c2c1727 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:19.990005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f757e-d128-4856-b3ea-4a555c2c1727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:19.995558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378979e4-3a27-4f64-b0e6-1afeda6d1c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:19.995558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378979e4-3a27-4f64-b0e6-1afeda6d1c91 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:19.995558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378979e4-3a27-4f64-b0e6-1afeda6d1c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:19.995558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378979e4-3a27-4f64-b0e6-1afeda6d1c91 None admin] PATH_INFO: `/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:19.996214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f757e-d128-4856-b3ea-4a555c2c1727 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:19.999518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378979e4-3a27-4f64-b0e6-1afeda6d1c91 None admin] RBAC: Authorizing `identity:get_user(user_id=b09585e90e3d40d2a2a3e9a1cdf34b08)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:20.001045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378979e4-3a27-4f64-b0e6-1afeda6d1c91 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:20.002801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 130/261] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:19 2022] GET /identity/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08 => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:20.010778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 132/262] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:20.012474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378979e4-3a27-4f64-b0e6-1afeda6d1c91 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:20.015924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f757e-d128-4856-b3ea-4a555c2c1727 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:20.181701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f757e-d128-4856-b3ea-4a555c2c1727 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:20.205656 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378979e4-3a27-4f64-b0e6-1afeda6d1c91 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:20.225250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c7702-e0d6-4ab8-b4dc-9241fa4fb316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:20.226192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c7702-e0d6-4ab8-b4dc-9241fa4fb316 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:20.226687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c7702-e0d6-4ab8-b4dc-9241fa4fb316 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:20.227155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c7702-e0d6-4ab8-b4dc-9241fa4fb316 None admin] PATH_INFO: `/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:20.228528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c7702-e0d6-4ab8-b4dc-9241fa4fb316 None admin] RBAC: Authorizing `identity:get_user(user_id=b09585e90e3d40d2a2a3e9a1cdf34b08)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:20.230214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c7702-e0d6-4ab8-b4dc-9241fa4fb316 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:20.232893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 133/263] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:20 2022] GET /identity/v3/users/b09585e90e3d40d2a2a3e9a1cdf34b08 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:20.238544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c7702-e0d6-4ab8-b4dc-9241fa4fb316 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:20.280658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea55009-3864-4533-b3b1-31a4fa5f101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:20.281657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea55009-3864-4533-b3b1-31a4fa5f101b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:20.282144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea55009-3864-4533-b3b1-31a4fa5f101b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:20.282636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea55009-3864-4533-b3b1-31a4fa5f101b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:20.299556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:20.375102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c7702-e0d6-4ab8-b4dc-9241fa4fb316 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:20.389773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:20.413327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f0b6c-2974-4b84-98ce-048e8c7eee29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:20.414307 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0b6c-2974-4b84-98ce-048e8c7eee29 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:20.414782 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0b6c-2974-4b84-98ce-048e8c7eee29 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:20.415245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0b6c-2974-4b84-98ce-048e8c7eee29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:20.467453 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea55009-3864-4533-b3b1-31a4fa5f101b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:20.468915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea55009-3864-4533-b3b1-31a4fa5f101b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:20.494541 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f0b6c-2974-4b84-98ce-048e8c7eee29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:20.496337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f0b6c-2974-4b84-98ce-048e8c7eee29 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:20.528084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:20.604722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 134/264] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:20.611601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f0b6c-2974-4b84-98ce-048e8c7eee29 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:20.700262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 131/265] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:20.701081 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea55009-3864-4533-b3b1-31a4fa5f101b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:20.758611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f0b6c-2974-4b84-98ce-048e8c7eee29 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:20.783256 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:20.837533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:20.850012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcef1c2-7724-483e-9952-5775293c3ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:20.850653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcef1c2-7724-483e-9952-5775293c3ddd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:20.850930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcef1c2-7724-483e-9952-5775293c3ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:20.851202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcef1c2-7724-483e-9952-5775293c3ddd None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:20.853359 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcef1c2-7724-483e-9952-5775293c3ddd None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:20.854822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcef1c2-7724-483e-9952-5775293c3ddd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:20.857040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 135/266] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:20 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:20.864034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea55009-3864-4533-b3b1-31a4fa5f101b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:20.864924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7fc6c-e488-42bb-9163-7979077d4265 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:20.865207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7fc6c-e488-42bb-9163-7979077d4265 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:20.865478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7fc6c-e488-42bb-9163-7979077d4265 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:20.866967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 136/267] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:20.869462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7fc6c-e488-42bb-9163-7979077d4265 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:20.921809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc0a67-e845-4999-b528-a88547f9fa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:20.922451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc0a67-e845-4999-b528-a88547f9fa15 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:20.922726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc0a67-e845-4999-b528-a88547f9fa15 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:20.922995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc0a67-e845-4999-b528-a88547f9fa15 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:20.924999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc0a67-e845-4999-b528-a88547f9fa15 None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:20.926445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc0a67-e845-4999-b528-a88547f9fa15 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:20.930917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 132/268] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:20 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:20.933028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8cd40-888e-4bd5-b413-4fde65e637a6 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:20.933299 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8cd40-888e-4bd5-b413-4fde65e637a6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:20.933569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8cd40-888e-4bd5-b413-4fde65e637a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:20.971249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-abe8cd40-888e-4bd5-b413-4fde65e637a6 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:20.972452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-abe8cd40-888e-4bd5-b413-4fde65e637a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:21.074831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7fc6c-e488-42bb-9163-7979077d4265 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:21.116268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 133/269] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:21.117271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8cd40-888e-4bd5-b413-4fde65e637a6 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:21.168785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbb8c1-7db2-467a-9182-85b5b55aaca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:21.170046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbb8c1-7db2-467a-9182-85b5b55aaca2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:21.170317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbb8c1-7db2-467a-9182-85b5b55aaca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:21.170581 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbb8c1-7db2-467a-9182-85b5b55aaca2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:21.171308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbb8c1-7db2-467a-9182-85b5b55aaca2 None admin] RBAC: Adding query filter params (role.id=43008618c1dc4cd49109f35c778080d3, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7, user.id=b09585e90e3d40d2a2a3e9a1cdf34b08) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:21.171635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbb8c1-7db2-467a-9182-85b5b55aaca2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:21.173078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbb8c1-7db2-467a-9182-85b5b55aaca2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:21.186088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 137/270] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:20 2022] GET /identity/v3/role_assignments?user.id=b09585e90e3d40d2a2a3e9a1cdf34b08&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=43008618c1dc4cd49109f35c778080d3 => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:21.197065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:21.245197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8cd40-888e-4bd5-b413-4fde65e637a6 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:21.283755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d731f00-0450-469f-b04d-c6a686421741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:21.284368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d731f00-0450-469f-b04d-c6a686421741 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:21.284644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d731f00-0450-469f-b04d-c6a686421741 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:21.284911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d731f00-0450-469f-b04d-c6a686421741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:21.285636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d731f00-0450-469f-b04d-c6a686421741 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7, user.id=b09585e90e3d40d2a2a3e9a1cdf34b08) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:21.285936 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d731f00-0450-469f-b04d-c6a686421741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:21.287379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d731f00-0450-469f-b04d-c6a686421741 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:21.294090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 134/271] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:21 2022] GET /identity/v3/role_assignments?user.id=b09585e90e3d40d2a2a3e9a1cdf34b08&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=705907b9bdec47cebf88b868a6a56356 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:21.363400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c6aa9-0423-4513-9b94-2f3386c671ca None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:21.363608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c6aa9-0423-4513-9b94-2f3386c671ca None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:21.363990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c6aa9-0423-4513-9b94-2f3386c671ca None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:21.365635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 138/272] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:21.371268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e5622-4e85-43f5-a995-a95d43be5c22 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:21.371869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e5622-4e85-43f5-a995-a95d43be5c22 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:21.372363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e5622-4e85-43f5-a995-a95d43be5c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:21.392815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-789e5622-4e85-43f5-a995-a95d43be5c22 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:21.394472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-789e5622-4e85-43f5-a995-a95d43be5c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:21.496877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 135/273] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:21.579725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e7920-a9a7-4bff-86d6-a765bb83ad69 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:21.580019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e7920-a9a7-4bff-86d6-a765bb83ad69 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:21.580277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e7920-a9a7-4bff-86d6-a765bb83ad69 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:21.581899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 139/274] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:21.587030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2045a-c7e3-40e7-ab23-2ed33beb8ba3 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:21.587586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2045a-c7e3-40e7-ab23-2ed33beb8ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:21.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2045a-c7e3-40e7-ab23-2ed33beb8ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:21.609738 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-a3c2045a-c7e3-40e7-ab23-2ed33beb8ba3 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:21.611280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a3c2045a-c7e3-40e7-ab23-2ed33beb8ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:21.649259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e7920-a9a7-4bff-86d6-a765bb83ad69 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:21.694625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:21.715454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 136/275] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:21.746852 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:21.791874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e7920-a9a7-4bff-86d6-a765bb83ad69 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:21.827502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9a22f-ff55-465d-9581-a6391636eca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:21.828185 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9a22f-ff55-465d-9581-a6391636eca9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:21.828461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9a22f-ff55-465d-9581-a6391636eca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:21.828733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9a22f-ff55-465d-9581-a6391636eca9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:21.839822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9a22f-ff55-465d-9581-a6391636eca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e5e9a22f-ff55-465d-9581-a6391636eca9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:21.845350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:21.846220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:21.847399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 140/276] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:21.852326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2045a-c7e3-40e7-ab23-2ed33beb8ba3 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:21.865673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9a22f-ff55-465d-9581-a6391636eca9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:21.969092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2045a-c7e3-40e7-ab23-2ed33beb8ba3 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:21.974270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9a22f-ff55-465d-9581-a6391636eca9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:21.998470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.007176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57349bfe-c962-4a54-85e8-b3c70397509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.008047 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57349bfe-c962-4a54-85e8-b3c70397509e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.011737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57349bfe-c962-4a54-85e8-b3c70397509e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.011737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57349bfe-c962-4a54-85e8-b3c70397509e None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.011737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57349bfe-c962-4a54-85e8-b3c70397509e None admin] RBAC: Adding query filter params (name=member) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:22.011737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57349bfe-c962-4a54-85e8-b3c70397509e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.012117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3b720-a7f6-4d10-ba34-087c82a5131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.012117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3b720-a7f6-4d10-ba34-087c82a5131a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.012117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3b720-a7f6-4d10-ba34-087c82a5131a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.012323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57349bfe-c962-4a54-85e8-b3c70397509e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.012391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3b720-a7f6-4d10-ba34-087c82a5131a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.013103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e3b720-a7f6-4d10-ba34-087c82a5131a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.014523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e3b720-a7f6-4d10-ba34-087c82a5131a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.016264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 141/277] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:22.016875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3b720-a7f6-4d10-ba34-087c82a5131a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.022255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 137/278] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:22.023283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57349bfe-c962-4a54-85e8-b3c70397509e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.081633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.120091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3b720-a7f6-4d10-ba34-087c82a5131a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.126147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57349bfe-c962-4a54-85e8-b3c70397509e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.152860 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b03625-af49-4311-ad9b-50a85419db95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.153515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b03625-af49-4311-ad9b-50a85419db95 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.153793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b03625-af49-4311-ad9b-50a85419db95 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.154066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b03625-af49-4311-ad9b-50a85419db95 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.158468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b91bc-7222-4c47-85a2-a5606ddc5193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.159068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b91bc-7222-4c47-85a2-a5606ddc5193 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.159355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b91bc-7222-4c47-85a2-a5606ddc5193 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.159656 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b91bc-7222-4c47-85a2-a5606ddc5193 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.160297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b91bc-7222-4c47-85a2-a5606ddc5193 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.161737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b91bc-7222-4c47-85a2-a5606ddc5193 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.163928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b03625-af49-4311-ad9b-50a85419db95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-d6b03625-af49-4311-ad9b-50a85419db95 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:22.169682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:22.170769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 142/279] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:22.171793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b03625-af49-4311-ad9b-50a85419db95 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.220385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.261936 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 138/280] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:09:22 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:22.263237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1307bd-5a77-48c2-b7af-aa455c9d67b3 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.263524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1307bd-5a77-48c2-b7af-aa455c9d67b3 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.263837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1307bd-5a77-48c2-b7af-aa455c9d67b3 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.265129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 139/281] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:22.265867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1307bd-5a77-48c2-b7af-aa455c9d67b3 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.277490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b03625-af49-4311-ad9b-50a85419db95 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.324220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa581034-7fed-4182-a717-22d0a48a50aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.324865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa581034-7fed-4182-a717-22d0a48a50aa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.325160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa581034-7fed-4182-a717-22d0a48a50aa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.325433 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa581034-7fed-4182-a717-22d0a48a50aa None admin] PATH_INFO: `/v3/users/59f9c230a2584899bc3eb395c11b75d6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.345879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa581034-7fed-4182-a717-22d0a48a50aa None admin] RBAC: Authorizing `identity:get_user(user_id=59f9c230a2584899bc3eb395c11b75d6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.347883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa581034-7fed-4182-a717-22d0a48a50aa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.349820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 143/282] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/users/59f9c230a2584899bc3eb395c11b75d6 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:22.351886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b7a7-4502-4279-8379-8561068862d9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.352803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b7a7-4502-4279-8379-8561068862d9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.353283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b7a7-4502-4279-8379-8561068862d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.377329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fe15b7a7-4502-4279-8379-8561068862d9 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:22.378167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1307bd-5a77-48c2-b7af-aa455c9d67b3 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.378668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fe15b7a7-4502-4279-8379-8561068862d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:22.392575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.420885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e2a2-ebfe-43af-aaee-d424d7175fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.421502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e2a2-ebfe-43af-aaee-d424d7175fee None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.421783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e2a2-ebfe-43af-aaee-d424d7175fee None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.422057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e2a2-ebfe-43af-aaee-d424d7175fee None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.423081 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608e2a2-ebfe-43af-aaee-d424d7175fee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:22.423081 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608e2a2-ebfe-43af-aaee-d424d7175fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.424607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608e2a2-ebfe-43af-aaee-d424d7175fee None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.436511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 140/283] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:22.437348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e2a2-ebfe-43af-aaee-d424d7175fee None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.460587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 144/284] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:22.461833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15b7a7-4502-4279-8379-8561068862d9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.512851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.565856 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e2a2-ebfe-43af-aaee-d424d7175fee None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.588979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15b7a7-4502-4279-8379-8561068862d9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.616724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39435b6c-b8dd-47d6-9b90-6ada5beff155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.617416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39435b6c-b8dd-47d6-9b90-6ada5beff155 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.617700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39435b6c-b8dd-47d6-9b90-6ada5beff155 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.617973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39435b6c-b8dd-47d6-9b90-6ada5beff155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.618776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39435b6c-b8dd-47d6-9b90-6ada5beff155 None admin] RBAC: Adding query filter params (role.id=0062155b0fbf41a19c9110ad71a046e3, user.id=59f9c230a2584899bc3eb395c11b75d6, scope.system=all) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:22.619095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39435b6c-b8dd-47d6-9b90-6ada5beff155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.620668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39435b6c-b8dd-47d6-9b90-6ada5beff155 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.626285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 141/285] 10.222.0.62 () {56 vars in 1308 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/role_assignments?user.id=59f9c230a2584899bc3eb395c11b75d6&scope.system=all&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:22.634424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca33e9-1178-412c-b754-8beca374fba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.635080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca33e9-1178-412c-b754-8beca374fba7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.635353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca33e9-1178-412c-b754-8beca374fba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.635719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca33e9-1178-412c-b754-8beca374fba7 None admin] PATH_INFO: `/v3/users/a8aa6abe8fb14b40b437943e2a36c3e5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.636978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca33e9-1178-412c-b754-8beca374fba7 None admin] RBAC: Authorizing `identity:get_user(user_id=a8aa6abe8fb14b40b437943e2a36c3e5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.638516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca33e9-1178-412c-b754-8beca374fba7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.643246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 145/286] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/users/a8aa6abe8fb14b40b437943e2a36c3e5 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:22.646442 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39435b6c-b8dd-47d6-9b90-6ada5beff155 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.693014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.756324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca33e9-1178-412c-b754-8beca374fba7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.774836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39435b6c-b8dd-47d6-9b90-6ada5beff155 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.824351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904af96-62d1-4558-9222-836d54672090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.825697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904af96-62d1-4558-9222-836d54672090 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.826077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904af96-62d1-4558-9222-836d54672090 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.826552 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904af96-62d1-4558-9222-836d54672090 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.827929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904af96-62d1-4558-9222-836d54672090 None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:22.829502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904af96-62d1-4558-9222-836d54672090 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:22.831647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 142/287] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:22.837676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904af96-62d1-4558-9222-836d54672090 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.892165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca33e9-1178-412c-b754-8beca374fba7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.900397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:22.929393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3787fc-33bd-48b4-832c-794eab6c5a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:22.931192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3787fc-33bd-48b4-832c-794eab6c5a17 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:22.931612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3787fc-33bd-48b4-832c-794eab6c5a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:22.931986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3787fc-33bd-48b4-832c-794eab6c5a17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:22.943171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3787fc-33bd-48b4-832c-794eab6c5a17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-8f3787fc-33bd-48b4-832c-794eab6c5a17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:22.948415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:22.949537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:22.951776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:22.952711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 146/288] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:22.961749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3787fc-33bd-48b4-832c-794eab6c5a17 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:22.964872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904af96-62d1-4558-9222-836d54672090 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:22.974460 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:23.004434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07eb340-d44c-4475-9a43-ae6a9cac5299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.005659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07eb340-d44c-4475-9a43-ae6a9cac5299 None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.006118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07eb340-d44c-4475-9a43-ae6a9cac5299 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.006449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07eb340-d44c-4475-9a43-ae6a9cac5299 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/users/a8aa6abe8fb14b40b437943e2a36c3e5/roles/b19408717eaf492c9999c46062b83ba8` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.009178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07eb340-d44c-4475-9a43-ae6a9cac5299 None admin] RBAC: Authorizing `identity:create_grant(project_id=88bda6b61b9c4485a3366fe286e2c1d7, user_id=a8aa6abe8fb14b40b437943e2a36c3e5, role_id=b19408717eaf492c9999c46062b83ba8)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.010694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07eb340-d44c-4475-9a43-ae6a9cac5299 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.074188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 143/289] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:09:22 2022] PUT /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/users/a8aa6abe8fb14b40b437943e2a36c3e5/roles/b19408717eaf492c9999c46062b83ba8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:09:23.116280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3787fc-33bd-48b4-832c-794eab6c5a17 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:23.154501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2836de-2d13-4975-a226-341896f06853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.155249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2836de-2d13-4975-a226-341896f06853 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.155402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2836de-2d13-4975-a226-341896f06853 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.155673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2836de-2d13-4975-a226-341896f06853 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.156309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2836de-2d13-4975-a226-341896f06853 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:23.156608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2836de-2d13-4975-a226-341896f06853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.157963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2836de-2d13-4975-a226-341896f06853 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.167187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 147/290] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:23.173012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07eb340-d44c-4475-9a43-ae6a9cac5299 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:23.303346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07eb340-d44c-4475-9a43-ae6a9cac5299 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:23.343610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec918a5-a844-40cf-bf15-4af2da13fab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.344253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec918a5-a844-40cf-bf15-4af2da13fab0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.344531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec918a5-a844-40cf-bf15-4af2da13fab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.344803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec918a5-a844-40cf-bf15-4af2da13fab0 None admin] PATH_INFO: `/v3/users/be7a1a21ccad4a6595a504abd0e43ce5` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.345934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec918a5-a844-40cf-bf15-4af2da13fab0 None admin] RBAC: Authorizing `identity:get_user(user_id=be7a1a21ccad4a6595a504abd0e43ce5)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.347372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec918a5-a844-40cf-bf15-4af2da13fab0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.348995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 144/291] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:23 2022] GET /identity/v3/users/be7a1a21ccad4a6595a504abd0e43ce5 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:23.354906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2836de-2d13-4975-a226-341896f06853 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:23.386498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:23.462925 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2836de-2d13-4975-a226-341896f06853 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:23.498049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ca27-333d-4af3-9306-effa5a9a29ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.498717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ca27-333d-4af3-9306-effa5a9a29ca None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.498994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ca27-333d-4af3-9306-effa5a9a29ca None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.499267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ca27-333d-4af3-9306-effa5a9a29ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.571846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5ca27-333d-4af3-9306-effa5a9a29ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.573219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5ca27-333d-4af3-9306-effa5a9a29ca None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.612069 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:23.703630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 148/292] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:23.709187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec918a5-a844-40cf-bf15-4af2da13fab0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:23.765298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:23.834726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec918a5-a844-40cf-bf15-4af2da13fab0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:23.870467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6426315-538a-4138-b76d-0542106debc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:23.871098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6426315-538a-4138-b76d-0542106debc0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:23.871374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6426315-538a-4138-b76d-0542106debc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:23.871681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6426315-538a-4138-b76d-0542106debc0 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:23.874066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6426315-538a-4138-b76d-0542106debc0 None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:23.875813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6426315-538a-4138-b76d-0542106debc0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:23.877898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 145/293] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:23 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:23.883429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ca27-333d-4af3-9306-effa5a9a29ca None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:24.025759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ca27-333d-4af3-9306-effa5a9a29ca None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:24.058815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:24.067288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235de191-4c24-4f85-8b24-7669f9e733d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:24.071566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235de191-4c24-4f85-8b24-7669f9e733d2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:24.071566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235de191-4c24-4f85-8b24-7669f9e733d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:24.071566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235de191-4c24-4f85-8b24-7669f9e733d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:24.071566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235de191-4c24-4f85-8b24-7669f9e733d2 None admin] RBAC: Adding query filter params (role.id=0062155b0fbf41a19c9110ad71a046e3, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7, user.id=be7a1a21ccad4a6595a504abd0e43ce5) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:24.071566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235de191-4c24-4f85-8b24-7669f9e733d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:24.071566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235de191-4c24-4f85-8b24-7669f9e733d2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:24.080179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 149/294] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:23 2022] GET /identity/v3/role_assignments?user.id=be7a1a21ccad4a6595a504abd0e43ce5&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:24.658992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9607f-f9ee-4d45-b449-13d6ea9d3fc4 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:24.659569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9607f-f9ee-4d45-b449-13d6ea9d3fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:24.659569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9607f-f9ee-4d45-b449-13d6ea9d3fc4 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:24.661122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 146/295] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:24.666132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f165d-17fa-4170-a8ef-8fdcf0e800b1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:24.666534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f165d-17fa-4170-a8ef-8fdcf0e800b1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:24.666721 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f165d-17fa-4170-a8ef-8fdcf0e800b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:24.689842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-139f165d-17fa-4170-a8ef-8fdcf0e800b1 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:24.691045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-139f165d-17fa-4170-a8ef-8fdcf0e800b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:24.769676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 150/296] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:24.916681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41208040-d385-4728-a55c-8571caec2c01 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:24.917047 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41208040-d385-4728-a55c-8571caec2c01 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:24.917422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41208040-d385-4728-a55c-8571caec2c01 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:24.919042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 147/297] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:24.924353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26ecb9-bf29-4053-92b4-0b3bb9041d6e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:24.924771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26ecb9-bf29-4053-92b4-0b3bb9041d6e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:24.925134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26ecb9-bf29-4053-92b4-0b3bb9041d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:24.944026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-da26ecb9-bf29-4053-92b4-0b3bb9041d6e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:24.945205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-da26ecb9-bf29-4053-92b4-0b3bb9041d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:25.000890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d932-9a5c-49fb-b769-e46d4c4f9208 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.001503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d932-9a5c-49fb-b769-e46d4c4f9208 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.001971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d932-9a5c-49fb-b769-e46d4c4f9208 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.004725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 148/298] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:09:25.008903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634b287-3b29-4bd5-8a2f-76b73336de10 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.009412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634b287-3b29-4bd5-8a2f-76b73336de10 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.009873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634b287-3b29-4bd5-8a2f-76b73336de10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.016549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 151/299] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:09:25.029918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c634b287-3b29-4bd5-8a2f-76b73336de10 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:25.031095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c634b287-3b29-4bd5-8a2f-76b73336de10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:25.042255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26ecb9-bf29-4053-92b4-0b3bb9041d6e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:25.102111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 149/300] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:25.123079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:25.159636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26ecb9-bf29-4053-92b4-0b3bb9041d6e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:25.193341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b2fd9-beaf-4a96-9dd9-76e7d674a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:25.193897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b2fd9-beaf-4a96-9dd9-76e7d674a147 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.194170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b2fd9-beaf-4a96-9dd9-76e7d674a147 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.194439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b2fd9-beaf-4a96-9dd9-76e7d674a147 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.205050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b2fd9-beaf-4a96-9dd9-76e7d674a147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-db8b2fd9-beaf-4a96-9dd9-76e7d674a147 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:25.209909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:25.211027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:25.212223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 152/301] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:25.220793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634b287-3b29-4bd5-8a2f-76b73336de10 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:25.251967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:25.319485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b2fd9-beaf-4a96-9dd9-76e7d674a147 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:25.324370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634b287-3b29-4bd5-8a2f-76b73336de10 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:25.359932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa4020-9ea7-4529-96d7-1fc6d8a31bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:25.361134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa4020-9ea7-4529-96d7-1fc6d8a31bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.361134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa4020-9ea7-4529-96d7-1fc6d8a31bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.361134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa4020-9ea7-4529-96d7-1fc6d8a31bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.361687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa4020-9ea7-4529-96d7-1fc6d8a31bc7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:25.361983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa4020-9ea7-4529-96d7-1fc6d8a31bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:25.363338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa4020-9ea7-4529-96d7-1fc6d8a31bc7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:25.373699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 150/302] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:25.378357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa4020-9ea7-4529-96d7-1fc6d8a31bc7 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:25.440823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b2fd9-beaf-4a96-9dd9-76e7d674a147 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:25.456748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:25.477009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a401896-a41d-4063-9b0f-002015dbb229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:25.477744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a401896-a41d-4063-9b0f-002015dbb229 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.478017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a401896-a41d-4063-9b0f-002015dbb229 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.478287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a401896-a41d-4063-9b0f-002015dbb229 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.479489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a401896-a41d-4063-9b0f-002015dbb229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:25.481216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a401896-a41d-4063-9b0f-002015dbb229 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:25.483659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 153/303] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:25.483965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a401896-a41d-4063-9b0f-002015dbb229 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:25.486250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:25.495253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa4020-9ea7-4529-96d7-1fc6d8a31bc7 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:25.541168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d43d9-1400-4908-a30c-6e1a117a17d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:25.541795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d43d9-1400-4908-a30c-6e1a117a17d1 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.542074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d43d9-1400-4908-a30c-6e1a117a17d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.542344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d43d9-1400-4908-a30c-6e1a117a17d1 None admin] PATH_INFO: `/v3/users/21b85784f18e4889a859a8c951f68bf6` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.555385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d43d9-1400-4908-a30c-6e1a117a17d1 None admin] RBAC: Authorizing `identity:get_user(user_id=21b85784f18e4889a859a8c951f68bf6)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:25.556847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d43d9-1400-4908-a30c-6e1a117a17d1 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:25.563884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 151/304] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:25 2022] GET /identity/v3/users/21b85784f18e4889a859a8c951f68bf6 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:25.564659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d43d9-1400-4908-a30c-6e1a117a17d1 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:25.592623 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a401896-a41d-4063-9b0f-002015dbb229 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:25.625875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea76b1d-db20-4b14-a083-39d532459684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:25.626490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76b1d-db20-4b14-a083-39d532459684 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.626764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76b1d-db20-4b14-a083-39d532459684 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.627033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76b1d-db20-4b14-a083-39d532459684 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.636805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea76b1d-db20-4b14-a083-39d532459684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-dea76b1d-db20-4b14-a083-39d532459684 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:25.642593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:25.643501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:25.644356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 154/305] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:25.645100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea76b1d-db20-4b14-a083-39d532459684 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:25.664929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d43d9-1400-4908-a30c-6e1a117a17d1 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:25.685963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:25.698292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a672b2b-23cb-49c2-becd-5b69927ccf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:25.698892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a672b2b-23cb-49c2-becd-5b69927ccf81 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.699173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a672b2b-23cb-49c2-becd-5b69927ccf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.699439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a672b2b-23cb-49c2-becd-5b69927ccf81 None admin] PATH_INFO: `/v3/groups` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.700078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a672b2b-23cb-49c2-becd-5b69927ccf81 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:25.702058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a672b2b-23cb-49c2-becd-5b69927ccf81 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:25.741176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea76b1d-db20-4b14-a083-39d532459684 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:25.773714 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11600e55-91aa-495d-845b-9349e4ee8d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:25.774306 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11600e55-91aa-495d-845b-9349e4ee8d8f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.774577 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11600e55-91aa-495d-845b-9349e4ee8d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.774844 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11600e55-91aa-495d-845b-9349e4ee8d8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.775572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11600e55-91aa-495d-845b-9349e4ee8d8f None admin] RBAC: Adding query filter params (role.id=b19408717eaf492c9999c46062b83ba8, user.id=21b85784f18e4889a859a8c951f68bf6, scope.system=all) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:25.775877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11600e55-91aa-495d-845b-9349e4ee8d8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:25.777286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11600e55-91aa-495d-845b-9349e4ee8d8f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:25.782037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 155/306] 10.222.0.62 () {56 vars in 1308 bytes} [Mon Sep 5 12:09:25 2022] GET /identity/v3/role_assignments?user.id=21b85784f18e4889a859a8c951f68bf6&scope.system=all&role.id=b19408717eaf492c9999c46062b83ba8 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:25.783127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75251c1a-eacf-402b-9cc4-b55cdd60f287 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.783405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75251c1a-eacf-402b-9cc4-b55cdd60f287 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.783706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75251c1a-eacf-402b-9cc4-b55cdd60f287 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.784883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 156/307] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:25.785585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75251c1a-eacf-402b-9cc4-b55cdd60f287 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:25.805506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 152/308] 10.222.0.62 () {60 vars in 1134 bytes} [Mon Sep 5 12:09:25 2022] POST /identity/v3/groups => generated 229 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:25.805506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad6e08-72d5-4b3c-892d-eac44766fa45 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.805506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad6e08-72d5-4b3c-892d-eac44766fa45 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.805506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad6e08-72d5-4b3c-892d-eac44766fa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.839864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-94ad6e08-72d5-4b3c-892d-eac44766fa45 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:25.841884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-94ad6e08-72d5-4b3c-892d-eac44766fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:25.846005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:25.921212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75251c1a-eacf-402b-9cc4-b55cdd60f287 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:25.935936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:25.950035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 153/309] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:25.972816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ff012-7490-4d7a-8e65-f4fa0036f0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:25.973476 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff012-7490-4d7a-8e65-f4fa0036f0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:25.973756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff012-7490-4d7a-8e65-f4fa0036f0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:25.974023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff012-7490-4d7a-8e65-f4fa0036f0d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:25.974738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff012-7490-4d7a-8e65-f4fa0036f0d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:25.975367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff012-7490-4d7a-8e65-f4fa0036f0d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:25.976914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff012-7490-4d7a-8e65-f4fa0036f0d5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:25.986795 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 157/310] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:25.993132 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad6e08-72d5-4b3c-892d-eac44766fa45 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:26.114632 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad6e08-72d5-4b3c-892d-eac44766fa45 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:26.149572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2ca8e-07a7-47bf-9eb6-9824e40b9b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:26.150213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2ca8e-07a7-47bf-9eb6-9824e40b9b02 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:26.150491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2ca8e-07a7-47bf-9eb6-9824e40b9b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:26.150814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2ca8e-07a7-47bf-9eb6-9824e40b9b02 None admin] PATH_INFO: `/v3/users/59f9c230a2584899bc3eb395c11b75d6` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:26.151969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2ca8e-07a7-47bf-9eb6-9824e40b9b02 None admin] RBAC: Authorizing `identity:get_user(user_id=59f9c230a2584899bc3eb395c11b75d6)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:26.153942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2ca8e-07a7-47bf-9eb6-9824e40b9b02 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:26.159868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 154/311] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:25 2022] GET /identity/v3/users/59f9c230a2584899bc3eb395c11b75d6 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:26.161419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ff012-7490-4d7a-8e65-f4fa0036f0d5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:26.251243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2ca8e-07a7-47bf-9eb6-9824e40b9b02 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:26.269413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:26.270225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ff012-7490-4d7a-8e65-f4fa0036f0d5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:26.309551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c072dd-50de-4751-92c7-60a6856d3708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:26.310243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c072dd-50de-4751-92c7-60a6856d3708 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:26.310524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c072dd-50de-4751-92c7-60a6856d3708 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:26.310801 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c072dd-50de-4751-92c7-60a6856d3708 None admin] PATH_INFO: `/v3/system/users/59f9c230a2584899bc3eb395c11b75d6/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:26.312548 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c072dd-50de-4751-92c7-60a6856d3708 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=59f9c230a2584899bc3eb395c11b75d6, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:26.314153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c072dd-50de-4751-92c7-60a6856d3708 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:26.375274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2ca8e-07a7-47bf-9eb6-9824e40b9b02 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:26.416617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 158/312] 10.222.0.62 () {58 vars in 1334 bytes} [Mon Sep 5 12:09:26 2022] PUT /identity/v3/system/users/59f9c230a2584899bc3eb395c11b75d6/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:26.417725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7f130-4ff4-4831-97a9-35be4e1fc407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:26.418454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7f130-4ff4-4831-97a9-35be4e1fc407 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:26.418763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7f130-4ff4-4831-97a9-35be4e1fc407 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:26.419666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7f130-4ff4-4831-97a9-35be4e1fc407 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:26.432966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7f130-4ff4-4831-97a9-35be4e1fc407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-b4a7f130-4ff4-4831-97a9-35be4e1fc407 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:26.438512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:26.439514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:26.440344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 155/313] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:26.445985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c072dd-50de-4751-92c7-60a6856d3708 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:26.483227 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:26.562723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c072dd-50de-4751-92c7-60a6856d3708 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:26.601879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0f254-ded5-47b1-b7ff-11f1d0b598de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:26.602515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0f254-ded5-47b1-b7ff-11f1d0b598de None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:26.602794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0f254-ded5-47b1-b7ff-11f1d0b598de None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:26.603066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0f254-ded5-47b1-b7ff-11f1d0b598de None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:26.604356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0f254-ded5-47b1-b7ff-11f1d0b598de None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:26.604356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0f254-ded5-47b1-b7ff-11f1d0b598de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:26.605390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0f254-ded5-47b1-b7ff-11f1d0b598de None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:26.614505 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 159/314] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:26.619943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7f130-4ff4-4831-97a9-35be4e1fc407 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:26.690626 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:26.726773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7f130-4ff4-4831-97a9-35be4e1fc407 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:26.764082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd281f2-45a2-4587-9f9c-2cdb66f4ffb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:26.764955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd281f2-45a2-4587-9f9c-2cdb66f4ffb9 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:26.765404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd281f2-45a2-4587-9f9c-2cdb66f4ffb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:26.765818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd281f2-45a2-4587-9f9c-2cdb66f4ffb9 None admin] PATH_INFO: `/v3/users/a8aa6abe8fb14b40b437943e2a36c3e5` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:26.767091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd281f2-45a2-4587-9f9c-2cdb66f4ffb9 None admin] RBAC: Authorizing `identity:get_user(user_id=a8aa6abe8fb14b40b437943e2a36c3e5)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:26.768744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd281f2-45a2-4587-9f9c-2cdb66f4ffb9 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:26.770781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 156/315] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:26 2022] GET /identity/v3/users/a8aa6abe8fb14b40b437943e2a36c3e5 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:26.776286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0f254-ded5-47b1-b7ff-11f1d0b598de None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:26.880792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:26.890025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0f254-ded5-47b1-b7ff-11f1d0b598de None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:26.924445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd93134-bc15-4e9c-b990-ca1cf67066ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:26.925054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93134-bc15-4e9c-b990-ca1cf67066ad None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:26.925325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93134-bc15-4e9c-b990-ca1cf67066ad None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:26.925591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93134-bc15-4e9c-b990-ca1cf67066ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:26.994879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd93134-bc15-4e9c-b990-ca1cf67066ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:26.996349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd93134-bc15-4e9c-b990-ca1cf67066ad None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:27.096115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:27.107107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 160/316] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:27.112811 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd281f2-45a2-4587-9f9c-2cdb66f4ffb9 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:27.218413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd281f2-45a2-4587-9f9c-2cdb66f4ffb9 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:27.252967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9c827-66c4-41a4-8ad6-4b1e2ab6eb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:27.253600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9c827-66c4-41a4-8ad6-4b1e2ab6eb7d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.253878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9c827-66c4-41a4-8ad6-4b1e2ab6eb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.254155 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9c827-66c4-41a4-8ad6-4b1e2ab6eb7d None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.255341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9c827-66c4-41a4-8ad6-4b1e2ab6eb7d None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:27.256840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9c827-66c4-41a4-8ad6-4b1e2ab6eb7d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:27.258777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 157/317] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:27 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:27.265274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd93134-bc15-4e9c-b990-ca1cf67066ad None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:27.371132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd93134-bc15-4e9c-b990-ca1cf67066ad None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:27.408090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07b699-4509-48ef-9b49-e7cf7e84bfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:27.408701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07b699-4509-48ef-9b49-e7cf7e84bfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.409002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07b699-4509-48ef-9b49-e7cf7e84bfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.409274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07b699-4509-48ef-9b49-e7cf7e84bfa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.410033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07b699-4509-48ef-9b49-e7cf7e84bfa6 None admin] RBAC: Adding query filter params (role.id=b19408717eaf492c9999c46062b83ba8, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7, user.id=a8aa6abe8fb14b40b437943e2a36c3e5) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:27.410338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07b699-4509-48ef-9b49-e7cf7e84bfa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:27.411787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07b699-4509-48ef-9b49-e7cf7e84bfa6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:27.418270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 161/318] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:27 2022] GET /identity/v3/role_assignments?user.id=a8aa6abe8fb14b40b437943e2a36c3e5&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=b19408717eaf492c9999c46062b83ba8 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:27.703441 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8f1a3-097e-432b-916a-c59aac385eea None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.703978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8f1a3-097e-432b-916a-c59aac385eea None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.703978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8f1a3-097e-432b-916a-c59aac385eea None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.705462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 158/319] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:27.710293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c7639-b231-425a-83ab-d0d47f69f4fe None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.710576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c7639-b231-425a-83ab-d0d47f69f4fe None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.711114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c7639-b231-425a-83ab-d0d47f69f4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.729645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-273c7639-b231-425a-83ab-d0d47f69f4fe None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:27.730805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-273c7639-b231-425a-83ab-d0d47f69f4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:27.781857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:27.799404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 162/320] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:27.901043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bda221-11d7-4070-bd92-2e961d9fc113 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.901043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bda221-11d7-4070-bd92-2e961d9fc113 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.901430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bda221-11d7-4070-bd92-2e961d9fc113 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.903251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 159/321] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:27.908944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce97ce-ec46-474c-9a7b-18d803cc086e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:27.909265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce97ce-ec46-474c-9a7b-18d803cc086e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:27.909501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce97ce-ec46-474c-9a7b-18d803cc086e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:27.928292 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-55ce97ce-ec46-474c-9a7b-18d803cc086e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:27.929472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-55ce97ce-ec46-474c-9a7b-18d803cc086e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:27.998277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 163/322] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:09:28.067268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bda221-11d7-4070-bd92-2e961d9fc113 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:28.071446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:28.170653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bda221-11d7-4070-bd92-2e961d9fc113 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:28.203910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb83dd0-72ee-4641-bdfa-31dbf4eb8838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:28.204589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb83dd0-72ee-4641-bdfa-31dbf4eb8838 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.204866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb83dd0-72ee-4641-bdfa-31dbf4eb8838 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.205186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb83dd0-72ee-4641-bdfa-31dbf4eb8838 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.215581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb83dd0-72ee-4641-bdfa-31dbf4eb8838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-2fb83dd0-72ee-4641-bdfa-31dbf4eb8838 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:28.221709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:28.222576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:28.223370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 160/323] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:28.230067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce97ce-ec46-474c-9a7b-18d803cc086e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:28.265675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb83dd0-72ee-4641-bdfa-31dbf4eb8838 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:28.331027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:28.336017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce97ce-ec46-474c-9a7b-18d803cc086e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:28.353670 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:28.371842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c335f-c923-4d33-94d8-aa1ee5038da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:28.373189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c335f-c923-4d33-94d8-aa1ee5038da9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.373622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c335f-c923-4d33-94d8-aa1ee5038da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.373892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c335f-c923-4d33-94d8-aa1ee5038da9 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.374559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c335f-c923-4d33-94d8-aa1ee5038da9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:28.374906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c335f-c923-4d33-94d8-aa1ee5038da9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:28.375200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb83dd0-72ee-4641-bdfa-31dbf4eb8838 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:28.376301 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c335f-c923-4d33-94d8-aa1ee5038da9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:28.384783 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 164/324] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:28.390786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c335f-c923-4d33-94d8-aa1ee5038da9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:28.414536 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe59bd1-7f1c-489a-a5fc-0c040fc20000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:28.415179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe59bd1-7f1c-489a-a5fc-0c040fc20000 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.415460 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe59bd1-7f1c-489a-a5fc-0c040fc20000 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.416197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe59bd1-7f1c-489a-a5fc-0c040fc20000 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.417322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe59bd1-7f1c-489a-a5fc-0c040fc20000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:28.418799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe59bd1-7f1c-489a-a5fc-0c040fc20000 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:28.420592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 161/325] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:28.425742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe59bd1-7f1c-489a-a5fc-0c040fc20000 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:28.495969 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c335f-c923-4d33-94d8-aa1ee5038da9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:28.528698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe59bd1-7f1c-489a-a5fc-0c040fc20000 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:28.529168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c981bae-ec2c-4e77-a882-2b5a67155d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:28.529810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c981bae-ec2c-4e77-a882-2b5a67155d65 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.530091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c981bae-ec2c-4e77-a882-2b5a67155d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.530357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c981bae-ec2c-4e77-a882-2b5a67155d65 None admin] PATH_INFO: `/v3/users/21b85784f18e4889a859a8c951f68bf6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.531577 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c981bae-ec2c-4e77-a882-2b5a67155d65 None admin] RBAC: Authorizing `identity:get_user(user_id=21b85784f18e4889a859a8c951f68bf6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:28.532995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c981bae-ec2c-4e77-a882-2b5a67155d65 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:28.534615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 165/326] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:28 2022] GET /identity/v3/users/21b85784f18e4889a859a8c951f68bf6 => generated 293 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:28.535804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba3927-403b-407b-8ecc-79e14a8a03aa None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.536086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba3927-403b-407b-8ecc-79e14a8a03aa None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.536354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba3927-403b-407b-8ecc-79e14a8a03aa None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.537544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 166/327] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:28.540207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba3927-403b-407b-8ecc-79e14a8a03aa None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:28.562026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:28.566674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df6481-af6b-45b8-ba46-e239f80e9d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:28.567273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df6481-af6b-45b8-ba46-e239f80e9d68 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.567569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df6481-af6b-45b8-ba46-e239f80e9d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.567849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df6481-af6b-45b8-ba46-e239f80e9d68 None admin] PATH_INFO: `/v3/groups` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.568683 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df6481-af6b-45b8-ba46-e239f80e9d68 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:28.569882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df6481-af6b-45b8-ba46-e239f80e9d68 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:28.588058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:28.641016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba3927-403b-407b-8ecc-79e14a8a03aa None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:28.673050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7095ad8-ebf8-4bc5-973f-147b36f84be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:28.673796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7095ad8-ebf8-4bc5-973f-147b36f84be1 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.674078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7095ad8-ebf8-4bc5-973f-147b36f84be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.674353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7095ad8-ebf8-4bc5-973f-147b36f84be1 None admin] PATH_INFO: `/v3/system/users/21b85784f18e4889a859a8c951f68bf6/roles/b19408717eaf492c9999c46062b83ba8` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.675771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7095ad8-ebf8-4bc5-973f-147b36f84be1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=21b85784f18e4889a859a8c951f68bf6, role_id=b19408717eaf492c9999c46062b83ba8)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:28.677121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7095ad8-ebf8-4bc5-973f-147b36f84be1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:28.684467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 162/328] 10.222.0.62 () {60 vars in 1134 bytes} [Mon Sep 5 12:09:28 2022] POST /identity/v3/groups => generated 226 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:09:28.686181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3bb60-62a3-4437-9105-38e73accf0c3 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:28.686455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3bb60-62a3-4437-9105-38e73accf0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:28.686722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3bb60-62a3-4437-9105-38e73accf0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:28.706375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-14d3bb60-62a3-4437-9105-38e73accf0c3 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:28.707648 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-14d3bb60-62a3-4437-9105-38e73accf0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:28.759786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 167/329] 10.222.0.62 () {58 vars in 1334 bytes} [Mon Sep 5 12:09:28 2022] PUT /identity/v3/system/users/21b85784f18e4889a859a8c951f68bf6/roles/b19408717eaf492c9999c46062b83ba8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:09:28.783563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 163/330] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:29.053385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7095ad8-ebf8-4bc5-973f-147b36f84be1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:29.164338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7095ad8-ebf8-4bc5-973f-147b36f84be1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:29.175675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:29.202132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1603d03-f2b2-4748-b681-73de42381e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:29.202777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1603d03-f2b2-4748-b681-73de42381e17 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:29.203054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1603d03-f2b2-4748-b681-73de42381e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:29.203324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1603d03-f2b2-4748-b681-73de42381e17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:29.215061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1603d03-f2b2-4748-b681-73de42381e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-a1603d03-f2b2-4748-b681-73de42381e17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:29.220474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:29.221344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:29.222117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 168/331] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:29.227876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3bb60-62a3-4437-9105-38e73accf0c3 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:29.341868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3bb60-62a3-4437-9105-38e73accf0c3 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:29.365445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:29.378285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126276a-2a1d-4866-9871-b516d769b956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:29.378905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126276a-2a1d-4866-9871-b516d769b956 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:29.379181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126276a-2a1d-4866-9871-b516d769b956 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:29.379452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126276a-2a1d-4866-9871-b516d769b956 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:29.380141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126276a-2a1d-4866-9871-b516d769b956 None admin] RBAC: Adding query filter params (name=member) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:29.380438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126276a-2a1d-4866-9871-b516d769b956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:29.381923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126276a-2a1d-4866-9871-b516d769b956 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:29.396966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 164/332] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:29.398600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1603d03-f2b2-4748-b681-73de42381e17 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:29.508177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1603d03-f2b2-4748-b681-73de42381e17 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:29.545193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c464af8-66ca-4d07-9b49-617d13cb3adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:29.545825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c464af8-66ca-4d07-9b49-617d13cb3adb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:29.546098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c464af8-66ca-4d07-9b49-617d13cb3adb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:29.546366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c464af8-66ca-4d07-9b49-617d13cb3adb None admin] PATH_INFO: `/v3/users/59f9c230a2584899bc3eb395c11b75d6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:29.547479 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c464af8-66ca-4d07-9b49-617d13cb3adb None admin] RBAC: Authorizing `identity:get_user(user_id=59f9c230a2584899bc3eb395c11b75d6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:29.549018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c464af8-66ca-4d07-9b49-617d13cb3adb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:29.550627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 169/333] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:29 2022] GET /identity/v3/users/59f9c230a2584899bc3eb395c11b75d6 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:29.556682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126276a-2a1d-4866-9871-b516d769b956 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:29.667139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126276a-2a1d-4866-9871-b516d769b956 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:29.701466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b10e74-c698-43a6-ae85-112c2b138cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:29.702125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b10e74-c698-43a6-ae85-112c2b138cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:29.702402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b10e74-c698-43a6-ae85-112c2b138cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:29.702670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b10e74-c698-43a6-ae85-112c2b138cd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:29.703390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b10e74-c698-43a6-ae85-112c2b138cd4 None admin] RBAC: Adding query filter params (role.id=0062155b0fbf41a19c9110ad71a046e3, user.id=59f9c230a2584899bc3eb395c11b75d6, scope.system=all) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:29.703716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b10e74-c698-43a6-ae85-112c2b138cd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:29.705157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b10e74-c698-43a6-ae85-112c2b138cd4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:29.723674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 165/334] 10.222.0.62 () {56 vars in 1308 bytes} [Mon Sep 5 12:09:29 2022] GET /identity/v3/role_assignments?user.id=59f9c230a2584899bc3eb395c11b75d6&scope.system=all&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 510 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:30.836133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32debd-df1c-433e-aa6c-059514003383 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:30.836642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32debd-df1c-433e-aa6c-059514003383 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:30.836718 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32debd-df1c-433e-aa6c-059514003383 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:30.838300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 170/335] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:30.844724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e747-a65a-4b95-8301-f6999e681d2b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:30.845176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e747-a65a-4b95-8301-f6999e681d2b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:30.845249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e747-a65a-4b95-8301-f6999e681d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:30.870372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-cde1e747-a65a-4b95-8301-f6999e681d2b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:30.871653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cde1e747-a65a-4b95-8301-f6999e681d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:30.875660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20edd0d-6f49-4956-91e5-04fb2355b2ab None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:30.876214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20edd0d-6f49-4956-91e5-04fb2355b2ab None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:30.876688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20edd0d-6f49-4956-91e5-04fb2355b2ab None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:30.878491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 171/336] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:30.883402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4999931-088f-438b-94cf-b669ae44e512 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:30.883989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4999931-088f-438b-94cf-b669ae44e512 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:30.884461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4999931-088f-438b-94cf-b669ae44e512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:30.912056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f4999931-088f-438b-94cf-b669ae44e512 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:30.913573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f4999931-088f-438b-94cf-b669ae44e512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:30.943924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:30.958930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 166/337] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:30.965517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:31.015365 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 172/338] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:31.150654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245620a-3b8d-4d21-9639-0c266f712587 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:31.151309 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245620a-3b8d-4d21-9639-0c266f712587 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:31.151824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245620a-3b8d-4d21-9639-0c266f712587 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:31.153355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 167/339] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:31.159357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c80aa2-f1ff-4baa-ad82-990099d8669c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:31.159866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c80aa2-f1ff-4baa-ad82-990099d8669c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:31.160330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c80aa2-f1ff-4baa-ad82-990099d8669c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:31.188520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b5c80aa2-f1ff-4baa-ad82-990099d8669c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:31.189972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b5c80aa2-f1ff-4baa-ad82-990099d8669c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:31.270925 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 173/340] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:31.352170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245620a-3b8d-4d21-9639-0c266f712587 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:31.479313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245620a-3b8d-4d21-9639-0c266f712587 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:31.483661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c80aa2-f1ff-4baa-ad82-990099d8669c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:31.549208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baeda4-6d95-4de4-985d-6ddc75591b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:31.550233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baeda4-6d95-4de4-985d-6ddc75591b4a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:31.550684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baeda4-6d95-4de4-985d-6ddc75591b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:31.551159 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baeda4-6d95-4de4-985d-6ddc75591b4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:31.568643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8baeda4-6d95-4de4-985d-6ddc75591b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-c8baeda4-6d95-4de4-985d-6ddc75591b4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:31.573978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:31.574865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 12:09:31.576267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:31.578495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 168/341] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:31.584390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baeda4-6d95-4de4-985d-6ddc75591b4a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:31.645285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c80aa2-f1ff-4baa-ad82-990099d8669c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:31.653532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:31.675983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:31.708448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e54d19-ecf1-4fe4-8ddd-9238a0af6ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:31.709109 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e54d19-ecf1-4fe4-8ddd-9238a0af6ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:31.709386 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e54d19-ecf1-4fe4-8ddd-9238a0af6ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:31.709657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e54d19-ecf1-4fe4-8ddd-9238a0af6ac2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:31.732405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e54d19-ecf1-4fe4-8ddd-9238a0af6ac2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-32e54d19-ecf1-4fe4-8ddd-9238a0af6ac2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:31.737246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:31.738168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:31.738955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 174/342] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:31.739635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e54d19-ecf1-4fe4-8ddd-9238a0af6ac2 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:31.750035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baeda4-6d95-4de4-985d-6ddc75591b4a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:31.824372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907da3c0-2140-4154-a9f6-d48215ebce32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:31.824983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907da3c0-2140-4154-a9f6-d48215ebce32 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:31.825257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907da3c0-2140-4154-a9f6-d48215ebce32 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:31.825561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907da3c0-2140-4154-a9f6-d48215ebce32 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:31.826408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907da3c0-2140-4154-a9f6-d48215ebce32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:31.826703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907da3c0-2140-4154-a9f6-d48215ebce32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:31.828824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907da3c0-2140-4154-a9f6-d48215ebce32 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:31.852305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 169/343] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:31.853928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907da3c0-2140-4154-a9f6-d48215ebce32 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:31.910324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e54d19-ecf1-4fe4-8ddd-9238a0af6ac2 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:31.953211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31292d-d3f4-4b64-82e5-57d3867d0369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:31.954214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31292d-d3f4-4b64-82e5-57d3867d0369 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:31.955489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31292d-d3f4-4b64-82e5-57d3867d0369 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:31.956092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31292d-d3f4-4b64-82e5-57d3867d0369 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:31.977180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31292d-d3f4-4b64-82e5-57d3867d0369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-4a31292d-d3f4-4b64-82e5-57d3867d0369 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:31.985743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:31.986680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:31.987489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:31.989911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 175/344] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:31.991029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31292d-d3f4-4b64-82e5-57d3867d0369 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:32.013471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:32.058063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:32.076776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907da3c0-2140-4154-a9f6-d48215ebce32 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:32.118172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31292d-d3f4-4b64-82e5-57d3867d0369 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:32.120899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648a427-a733-452f-a195-6f9bea9cd087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:32.122094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648a427-a733-452f-a195-6f9bea9cd087 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:32.122094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648a427-a733-452f-a195-6f9bea9cd087 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:32.122094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648a427-a733-452f-a195-6f9bea9cd087 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:32.123226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648a427-a733-452f-a195-6f9bea9cd087 None admin] RBAC: Adding query filter params (name=member) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:32.123528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648a427-a733-452f-a195-6f9bea9cd087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:32.125084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648a427-a733-452f-a195-6f9bea9cd087 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:32.134063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 170/345] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:32.134699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648a427-a733-452f-a195-6f9bea9cd087 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:32.160511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b31281-0740-4116-a4ff-eac1dc6db280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:32.161155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b31281-0740-4116-a4ff-eac1dc6db280 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:32.161435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b31281-0740-4116-a4ff-eac1dc6db280 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:32.161708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b31281-0740-4116-a4ff-eac1dc6db280 None admin] PATH_INFO: `/v3/users/21b85784f18e4889a859a8c951f68bf6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:32.163077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b31281-0740-4116-a4ff-eac1dc6db280 None admin] RBAC: Authorizing `identity:get_user(user_id=21b85784f18e4889a859a8c951f68bf6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:32.164724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b31281-0740-4116-a4ff-eac1dc6db280 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:32.166979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 176/346] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:09:31 2022] GET /identity/v3/users/21b85784f18e4889a859a8c951f68bf6 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:32.168325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d54ec1-540c-4ac3-bde2-f6c561961aad None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:32.168609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d54ec1-540c-4ac3-bde2-f6c561961aad None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:32.168885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d54ec1-540c-4ac3-bde2-f6c561961aad None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:32.171708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 177/347] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:32.172894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d54ec1-540c-4ac3-bde2-f6c561961aad None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:32.267444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648a427-a733-452f-a195-6f9bea9cd087 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:32.282165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:32.303670 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d54ec1-540c-4ac3-bde2-f6c561961aad None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:32.312585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cac27-a843-4288-936f-c14d891ad332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:32.313277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cac27-a843-4288-936f-c14d891ad332 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:32.313698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cac27-a843-4288-936f-c14d891ad332 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:32.313885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cac27-a843-4288-936f-c14d891ad332 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:32.314544 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cac27-a843-4288-936f-c14d891ad332 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:32.314845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cac27-a843-4288-936f-c14d891ad332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:32.316356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cac27-a843-4288-936f-c14d891ad332 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:32.325089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 171/348] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:32.326646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13ea9e-0630-4bda-bd8b-f60d94a87c33 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:32.326930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13ea9e-0630-4bda-bd8b-f60d94a87c33 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:32.327202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13ea9e-0630-4bda-bd8b-f60d94a87c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:32.348330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0dce8-bdd6-47f3-bef1-fcc51794c6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:32.349408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0dce8-bdd6-47f3-bef1-fcc51794c6bc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:32.349979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0dce8-bdd6-47f3-bef1-fcc51794c6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:32.350418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0dce8-bdd6-47f3-bef1-fcc51794c6bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:32.356812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-cb13ea9e-0630-4bda-bd8b-f60d94a87c33 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:32.358037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cb13ea9e-0630-4bda-bd8b-f60d94a87c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:32.374038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:32.441639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 172/349] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:32.442926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13ea9e-0630-4bda-bd8b-f60d94a87c33 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:32.446151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0dce8-bdd6-47f3-bef1-fcc51794c6bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:32.447692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0dce8-bdd6-47f3-bef1-fcc51794c6bc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:32.555254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13ea9e-0630-4bda-bd8b-f60d94a87c33 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:32.560659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:32.579930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 178/350] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:32.583036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0dce8-bdd6-47f3-bef1-fcc51794c6bc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:32.595624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cb905-4b60-4b9a-91ab-3ce4c9aa884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:32.596528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cb905-4b60-4b9a-91ab-3ce4c9aa884a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:32.596995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cb905-4b60-4b9a-91ab-3ce4c9aa884a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:32.597479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cb905-4b60-4b9a-91ab-3ce4c9aa884a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:32.598434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cb905-4b60-4b9a-91ab-3ce4c9aa884a None admin] RBAC: Adding query filter params (role.id=b19408717eaf492c9999c46062b83ba8, user.id=21b85784f18e4889a859a8c951f68bf6, scope.system=all) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:32.598944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cb905-4b60-4b9a-91ab-3ce4c9aa884a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:32.600684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cb905-4b60-4b9a-91ab-3ce4c9aa884a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:32.613624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 173/351] 10.222.0.62 () {56 vars in 1308 bytes} [Mon Sep 5 12:09:32 2022] GET /identity/v3/role_assignments?user.id=21b85784f18e4889a859a8c951f68bf6&scope.system=all&role.id=b19408717eaf492c9999c46062b83ba8 => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:32.614700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cb905-4b60-4b9a-91ab-3ce4c9aa884a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:32.624649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:32.728003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cb905-4b60-4b9a-91ab-3ce4c9aa884a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:32.741610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0dce8-bdd6-47f3-bef1-fcc51794c6bc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:32.773754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e5397-5f94-4fb5-b194-f96a60465359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:32.774789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e5397-5f94-4fb5-b194-f96a60465359 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:32.775258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e5397-5f94-4fb5-b194-f96a60465359 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:32.775741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e5397-5f94-4fb5-b194-f96a60465359 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:32.780444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e5397-5f94-4fb5-b194-f96a60465359 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:32.782298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e5397-5f94-4fb5-b194-f96a60465359 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:32.784266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 174/352] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:32 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:32.791625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e5397-5f94-4fb5-b194-f96a60465359 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:32.794707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4bcd97-ad60-4dfc-8522-368afc749ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:32.795329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bcd97-ad60-4dfc-8522-368afc749ace None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:32.795627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bcd97-ad60-4dfc-8522-368afc749ace None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:32.795899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bcd97-ad60-4dfc-8522-368afc749ace None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:32.893211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:32.921336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4bcd97-ad60-4dfc-8522-368afc749ace None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:32.922800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4bcd97-ad60-4dfc-8522-368afc749ace None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:32.951807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:32.979576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e5397-5f94-4fb5-b194-f96a60465359 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:33.024011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed914-b87c-4ace-b18d-0a4a98fdf8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:33.024661 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed914-b87c-4ace-b18d-0a4a98fdf8ff None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.024939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed914-b87c-4ace-b18d-0a4a98fdf8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.025211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed914-b87c-4ace-b18d-0a4a98fdf8ff None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.034240 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ed914-b87c-4ace-b18d-0a4a98fdf8ff None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:33.035768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ed914-b87c-4ace-b18d-0a4a98fdf8ff None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:33.041070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 175/353] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:32 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:33.041070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed914-b87c-4ace-b18d-0a4a98fdf8ff None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:33.119131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 179/354] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:33.120076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4bcd97-ad60-4dfc-8522-368afc749ace None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:33.177883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed914-b87c-4ace-b18d-0a4a98fdf8ff None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:33.212195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:33.219883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c98dec-0fe3-4fbd-ae2a-15bef321548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:33.220968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c98dec-0fe3-4fbd-ae2a-15bef321548d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.221527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c98dec-0fe3-4fbd-ae2a-15bef321548d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.222017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c98dec-0fe3-4fbd-ae2a-15bef321548d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.226867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4bcd97-ad60-4dfc-8522-368afc749ace None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:33.236696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:33.241897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c98dec-0fe3-4fbd-ae2a-15bef321548d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-81c98dec-0fe3-4fbd-ae2a-15bef321548d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:33.247144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:33.248641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:33.249451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:33.251757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 176/355] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:33.252811 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c98dec-0fe3-4fbd-ae2a-15bef321548d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:33.268499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5767cf-2b1a-449a-a3db-4c30509b78b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:33.269128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5767cf-2b1a-449a-a3db-4c30509b78b0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.282450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5767cf-2b1a-449a-a3db-4c30509b78b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.282450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5767cf-2b1a-449a-a3db-4c30509b78b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.282450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5767cf-2b1a-449a-a3db-4c30509b78b0 None admin] RBAC: Adding query filter params (group.id=ce8794bed1234e3e98a2099b19c9ce67, role.id=0062155b0fbf41a19c9110ad71a046e3, scope.project.id=04853db2b388437e916da478e79b0816) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:33.282450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5767cf-2b1a-449a-a3db-4c30509b78b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:33.282450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5767cf-2b1a-449a-a3db-4c30509b78b0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:33.282821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 180/356] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Sep 5 12:09:33 2022] GET /identity/v3/role_assignments?group.id=ce8794bed1234e3e98a2099b19c9ce67&scope.project.id=04853db2b388437e916da478e79b0816&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:33.282821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5767cf-2b1a-449a-a3db-4c30509b78b0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:33.405355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5767cf-2b1a-449a-a3db-4c30509b78b0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:33.441362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c98dec-0fe3-4fbd-ae2a-15bef321548d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:33.448725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16db6-da54-4766-af20-4d341064b116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:33.449351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16db6-da54-4766-af20-4d341064b116 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.449623 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16db6-da54-4766-af20-4d341064b116 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.449905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16db6-da54-4766-af20-4d341064b116 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.450604 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16db6-da54-4766-af20-4d341064b116 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:33.451486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16db6-da54-4766-af20-4d341064b116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:33.452907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16db6-da54-4766-af20-4d341064b116 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:33.464732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 181/357] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:33.466434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6a262-93a9-4ab8-a6e2-3e2cf0d884f1 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.466717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6a262-93a9-4ab8-a6e2-3e2cf0d884f1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.466988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6a262-93a9-4ab8-a6e2-3e2cf0d884f1 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.473965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 182/358] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:33.474833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6a262-93a9-4ab8-a6e2-3e2cf0d884f1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:33.493374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126e585-fb36-497c-b925-a483242ec583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:33.494024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126e585-fb36-497c-b925-a483242ec583 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.494303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126e585-fb36-497c-b925-a483242ec583 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.494570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126e585-fb36-497c-b925-a483242ec583 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.495931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126e585-fb36-497c-b925-a483242ec583 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:33.498171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126e585-fb36-497c-b925-a483242ec583 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:33.500137 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 177/359] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:33 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:33.501607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:33.502721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d5c4b-8c69-47d9-970c-0748e051a4f8 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.503197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d5c4b-8c69-47d9-970c-0748e051a4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.503702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d5c4b-8c69-47d9-970c-0748e051a4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.525652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-289d5c4b-8c69-47d9-970c-0748e051a4f8 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:33.527145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-289d5c4b-8c69-47d9-970c-0748e051a4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:33.601444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:33.613349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 178/360] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:33.613652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d5c4b-8c69-47d9-970c-0748e051a4f8 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:33.624813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6a262-93a9-4ab8-a6e2-3e2cf0d884f1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:33.661710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf271a-adbd-4c5e-8452-4a700331367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:33.662319 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf271a-adbd-4c5e-8452-4a700331367b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.662594 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf271a-adbd-4c5e-8452-4a700331367b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.662860 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf271a-adbd-4c5e-8452-4a700331367b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.748779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf271a-adbd-4c5e-8452-4a700331367b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:33.750174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf271a-adbd-4c5e-8452-4a700331367b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:33.757916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d5c4b-8c69-47d9-970c-0748e051a4f8 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:33.768514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:33.802781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da3755-920d-4887-afc0-f68da7e80612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:33.803709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da3755-920d-4887-afc0-f68da7e80612 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.803989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da3755-920d-4887-afc0-f68da7e80612 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.804306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da3755-920d-4887-afc0-f68da7e80612 None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.805891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da3755-920d-4887-afc0-f68da7e80612 None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:33.807353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da3755-920d-4887-afc0-f68da7e80612 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:33.809880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 179/361] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:33 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:33.811117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4deb2-4282-4db7-8258-c8189b8d5bd8 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.811400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4deb2-4282-4db7-8258-c8189b8d5bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.811694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4deb2-4282-4db7-8258-c8189b8d5bd8 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.813174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 180/362] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:33.821297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dac8af-f774-4241-97f1-eb991f18af82 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:33.821584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dac8af-f774-4241-97f1-eb991f18af82 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:33.821857 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dac8af-f774-4241-97f1-eb991f18af82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:33.851761 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-53dac8af-f774-4241-97f1-eb991f18af82 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:33.853321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-53dac8af-f774-4241-97f1-eb991f18af82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:33.887489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 183/363] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:33.888973 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf271a-adbd-4c5e-8452-4a700331367b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:33.924813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:33.938898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 181/364] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:33.944256 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dac8af-f774-4241-97f1-eb991f18af82 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:34.019294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf271a-adbd-4c5e-8452-4a700331367b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:34.037030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:34.073339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e837ef-75a5-45ba-b6fb-8b9a7c7b51f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:34.074419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e837ef-75a5-45ba-b6fb-8b9a7c7b51f6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:34.074709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e837ef-75a5-45ba-b6fb-8b9a7c7b51f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:34.075026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e837ef-75a5-45ba-b6fb-8b9a7c7b51f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:34.076655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e837ef-75a5-45ba-b6fb-8b9a7c7b51f6 None admin] RBAC: Adding query filter params (group.id=ce8794bed1234e3e98a2099b19c9ce67, role.id=43008618c1dc4cd49109f35c778080d3, scope.project.id=04853db2b388437e916da478e79b0816) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:34.076966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e837ef-75a5-45ba-b6fb-8b9a7c7b51f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:34.078409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e837ef-75a5-45ba-b6fb-8b9a7c7b51f6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:34.078835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dac8af-f774-4241-97f1-eb991f18af82 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:34.089510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 184/365] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Sep 5 12:09:33 2022] GET /identity/v3/role_assignments?group.id=ce8794bed1234e3e98a2099b19c9ce67&scope.project.id=04853db2b388437e916da478e79b0816&role.id=43008618c1dc4cd49109f35c778080d3 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:34.089510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e837ef-75a5-45ba-b6fb-8b9a7c7b51f6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:34.157620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9582a-0859-4263-ba51-7822867f7fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:34.158278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9582a-0859-4263-ba51-7822867f7fee None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:34.158557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9582a-0859-4263-ba51-7822867f7fee None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:34.158828 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9582a-0859-4263-ba51-7822867f7fee None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:34.160344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9582a-0859-4263-ba51-7822867f7fee None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:34.164231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9582a-0859-4263-ba51-7822867f7fee None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:34.168766 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 182/366] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:33 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:34.177210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9582a-0859-4263-ba51-7822867f7fee None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:34.259454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:34.307231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9582a-0859-4263-ba51-7822867f7fee None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:34.309142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e837ef-75a5-45ba-b6fb-8b9a7c7b51f6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:34.349877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa6ff0-49a0-4b7d-9331-3b61474ac25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:34.350547 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa6ff0-49a0-4b7d-9331-3b61474ac25f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:34.350833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa6ff0-49a0-4b7d-9331-3b61474ac25f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:34.351107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa6ff0-49a0-4b7d-9331-3b61474ac25f None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:34.352315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa6ff0-49a0-4b7d-9331-3b61474ac25f None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:34.353772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa6ff0-49a0-4b7d-9331-3b61474ac25f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:34.356417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 183/367] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:34 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:34.357391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa6ff0-49a0-4b7d-9331-3b61474ac25f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:34.382373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b7c9e-9142-44ce-9cbe-36cfc4b5cc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:34.383356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b7c9e-9142-44ce-9cbe-36cfc4b5cc3e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:34.384436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b7c9e-9142-44ce-9cbe-36cfc4b5cc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:34.384909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b7c9e-9142-44ce-9cbe-36cfc4b5cc3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:34.402226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b7c9e-9142-44ce-9cbe-36cfc4b5cc3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-df2b7c9e-9142-44ce-9cbe-36cfc4b5cc3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:34.409273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:34.410204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:34.411015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:34.416121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 185/368] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:34.416121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b7c9e-9142-44ce-9cbe-36cfc4b5cc3e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:34.476113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:34.503925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa6ff0-49a0-4b7d-9331-3b61474ac25f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:34.535457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:34.541966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523148ad-f20c-4013-ba76-df647cb952ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:34.542868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523148ad-f20c-4013-ba76-df647cb952ad None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:34.543342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523148ad-f20c-4013-ba76-df647cb952ad None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:34.543834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523148ad-f20c-4013-ba76-df647cb952ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:34.553552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b7c9e-9142-44ce-9cbe-36cfc4b5cc3e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:34.555776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523148ad-f20c-4013-ba76-df647cb952ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-523148ad-f20c-4013-ba76-df647cb952ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:34.561892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:34.562832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:34.563671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:34.567355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 184/369] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:34.568594 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523148ad-f20c-4013-ba76-df647cb952ad None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:34.596236 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136fa92-844f-4122-ba6b-077874bdc659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:34.596898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136fa92-844f-4122-ba6b-077874bdc659 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:34.597234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136fa92-844f-4122-ba6b-077874bdc659 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:34.597511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136fa92-844f-4122-ba6b-077874bdc659 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:34.598487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136fa92-844f-4122-ba6b-077874bdc659 None admin] RBAC: Adding query filter params (group.id=ce8794bed1234e3e98a2099b19c9ce67, role.id=0062155b0fbf41a19c9110ad71a046e3, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:34.598616 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136fa92-844f-4122-ba6b-077874bdc659 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:34.600059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136fa92-844f-4122-ba6b-077874bdc659 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:34.606763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 186/370] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Sep 5 12:09:34 2022] GET /identity/v3/role_assignments?group.id=ce8794bed1234e3e98a2099b19c9ce67&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:34.608752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136fa92-844f-4122-ba6b-077874bdc659 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:34.682872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523148ad-f20c-4013-ba76-df647cb952ad None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:34.722299 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff3a9f-9a81-4e75-9fb2-a76ee6133ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:34.723252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff3a9f-9a81-4e75-9fb2-a76ee6133ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:34.723758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff3a9f-9a81-4e75-9fb2-a76ee6133ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:34.724247 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff3a9f-9a81-4e75-9fb2-a76ee6133ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:34.725193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff3a9f-9a81-4e75-9fb2-a76ee6133ed4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:34.725537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:34.725901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff3a9f-9a81-4e75-9fb2-a76ee6133ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:34.727468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff3a9f-9a81-4e75-9fb2-a76ee6133ed4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:34.729193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136fa92-844f-4122-ba6b-077874bdc659 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:34.737899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 185/371] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:34.761288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff3a9f-9a81-4e75-9fb2-a76ee6133ed4 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:34.775630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374addc-00f1-4241-b798-35b8bccc7e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:34.776262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374addc-00f1-4241-b798-35b8bccc7e29 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:34.777133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374addc-00f1-4241-b798-35b8bccc7e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:34.777133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374addc-00f1-4241-b798-35b8bccc7e29 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:34.777581 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374addc-00f1-4241-b798-35b8bccc7e29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:34.777883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374addc-00f1-4241-b798-35b8bccc7e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:34.779406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374addc-00f1-4241-b798-35b8bccc7e29 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:34.796174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 187/372] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:34.807991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374addc-00f1-4241-b798-35b8bccc7e29 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:34.835837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:34.890171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff3a9f-9a81-4e75-9fb2-a76ee6133ed4 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:34.930187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374addc-00f1-4241-b798-35b8bccc7e29 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:34.930536 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238b1cd-05ed-440c-a729-8617c3cf69b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:34.931378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238b1cd-05ed-440c-a729-8617c3cf69b2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:34.932038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238b1cd-05ed-440c-a729-8617c3cf69b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:34.933324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238b1cd-05ed-440c-a729-8617c3cf69b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:34.969778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98000100-d949-45ff-8556-7813ebbcfaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:34.970913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98000100-d949-45ff-8556-7813ebbcfaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:34.971385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98000100-d949-45ff-8556-7813ebbcfaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:34.971808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98000100-d949-45ff-8556-7813ebbcfaa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.014585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238b1cd-05ed-440c-a729-8617c3cf69b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:35.016451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238b1cd-05ed-440c-a729-8617c3cf69b2 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:35.027983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:35.052142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98000100-d949-45ff-8556-7813ebbcfaa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:35.054021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98000100-d949-45ff-8556-7813ebbcfaa7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:35.073468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:35.125665 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 186/373] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:09:35.136538 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238b1cd-05ed-440c-a729-8617c3cf69b2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:35.186141 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 188/374] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:35.192078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98000100-d949-45ff-8556-7813ebbcfaa7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:35.267748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:35.316775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98000100-d949-45ff-8556-7813ebbcfaa7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:35.346971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238b1cd-05ed-440c-a729-8617c3cf69b2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:35.363046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72e5e3-91d8-40dd-814e-21b4cb8e5d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:35.364038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72e5e3-91d8-40dd-814e-21b4cb8e5d2f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:35.364522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72e5e3-91d8-40dd-814e-21b4cb8e5d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:35.365020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72e5e3-91d8-40dd-814e-21b4cb8e5d2f None admin] PATH_INFO: `/v3/projects/d80f282fe9da41e0b08a14ba58840408` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.367294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72e5e3-91d8-40dd-814e-21b4cb8e5d2f None admin] RBAC: Authorizing `identity:get_project(project_id=d80f282fe9da41e0b08a14ba58840408)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:35.368970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72e5e3-91d8-40dd-814e-21b4cb8e5d2f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:35.371563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 189/375] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:35 2022] GET /identity/v3/projects/d80f282fe9da41e0b08a14ba58840408 => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:35.380951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72e5e3-91d8-40dd-814e-21b4cb8e5d2f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:35.413439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cd8b4-7aaa-4e5d-b78a-e96242efcc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:35.414096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cd8b4-7aaa-4e5d-b78a-e96242efcc17 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:35.414403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cd8b4-7aaa-4e5d-b78a-e96242efcc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:35.414624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cd8b4-7aaa-4e5d-b78a-e96242efcc17 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.416802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cd8b4-7aaa-4e5d-b78a-e96242efcc17 None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:35.418362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cd8b4-7aaa-4e5d-b78a-e96242efcc17 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:35.423645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 187/376] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:35 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:35.449266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cd8b4-7aaa-4e5d-b78a-e96242efcc17 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:35.490742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:35.512792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72e5e3-91d8-40dd-814e-21b4cb8e5d2f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:35.548880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:35.556736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30f632-a5ef-428b-b37c-860302b9a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:35.557959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30f632-a5ef-428b-b37c-860302b9a43a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:35.558407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30f632-a5ef-428b-b37c-860302b9a43a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:35.558843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30f632-a5ef-428b-b37c-860302b9a43a None admin] PATH_INFO: `/v3/groups/5689a76b675d494baa09a9faf3795f04` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.566063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30f632-a5ef-428b-b37c-860302b9a43a None admin] RBAC: Authorizing `identity:get_group(group_id=5689a76b675d494baa09a9faf3795f04)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:35.568073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30f632-a5ef-428b-b37c-860302b9a43a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:35.570675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 190/377] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:35 2022] GET /identity/v3/groups/5689a76b675d494baa09a9faf3795f04 => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:35.580332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30f632-a5ef-428b-b37c-860302b9a43a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:35.636055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cd8b4-7aaa-4e5d-b78a-e96242efcc17 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:35.678601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601250ef-c6a1-4fd7-9df0-bffa93135aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:35.679575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601250ef-c6a1-4fd7-9df0-bffa93135aac None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:35.680314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601250ef-c6a1-4fd7-9df0-bffa93135aac None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:35.682855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601250ef-c6a1-4fd7-9df0-bffa93135aac None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.684467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601250ef-c6a1-4fd7-9df0-bffa93135aac None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:35.686178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601250ef-c6a1-4fd7-9df0-bffa93135aac None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:35.688555 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 188/378] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:35 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:35.695399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601250ef-c6a1-4fd7-9df0-bffa93135aac None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:35.770529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30f632-a5ef-428b-b37c-860302b9a43a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:35.803251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:35.816252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eef880-e7e0-4621-a84e-09a8f7bb5e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:35.826894 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601250ef-c6a1-4fd7-9df0-bffa93135aac None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:35.827045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eef880-e7e0-4621-a84e-09a8f7bb5e9e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:35.827045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eef880-e7e0-4621-a84e-09a8f7bb5e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:35.827045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eef880-e7e0-4621-a84e-09a8f7bb5e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.827045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eef880-e7e0-4621-a84e-09a8f7bb5e9e None admin] RBAC: Adding query filter params (group.id=5689a76b675d494baa09a9faf3795f04, role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=d80f282fe9da41e0b08a14ba58840408) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:35.827045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eef880-e7e0-4621-a84e-09a8f7bb5e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:35.827045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eef880-e7e0-4621-a84e-09a8f7bb5e9e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:35.828972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 191/379] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Sep 5 12:09:35 2022] GET /identity/v3/role_assignments?group.id=5689a76b675d494baa09a9faf3795f04&scope.project.id=d80f282fe9da41e0b08a14ba58840408&role.id=705907b9bdec47cebf88b868a6a56356 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:35.830425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ffafe-146c-4d22-ac35-868e7988da02 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:35.830867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ffafe-146c-4d22-ac35-868e7988da02 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:35.831314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ffafe-146c-4d22-ac35-868e7988da02 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.833436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 192/380] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:35.838946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e16e57-13ef-4c14-bcf8-0ee7afa15bd4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:35.839204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e16e57-13ef-4c14-bcf8-0ee7afa15bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:35.839468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e16e57-13ef-4c14-bcf8-0ee7afa15bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.863936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-57e16e57-13ef-4c14-bcf8-0ee7afa15bd4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:35.865166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-57e16e57-13ef-4c14-bcf8-0ee7afa15bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:35.876551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2601fb0-1011-42d9-959d-8586387d115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:35.877541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2601fb0-1011-42d9-959d-8586387d115c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:35.878013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2601fb0-1011-42d9-959d-8586387d115c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:35.878511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2601fb0-1011-42d9-959d-8586387d115c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:35.879423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2601fb0-1011-42d9-959d-8586387d115c None admin] RBAC: Adding query filter params (group.id=ce8794bed1234e3e98a2099b19c9ce67, role.id=43008618c1dc4cd49109f35c778080d3, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:35.880461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2601fb0-1011-42d9-959d-8586387d115c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:35.882465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2601fb0-1011-42d9-959d-8586387d115c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:35.895954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 189/381] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Sep 5 12:09:35 2022] GET /identity/v3/role_assignments?group.id=ce8794bed1234e3e98a2099b19c9ce67&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=43008618c1dc4cd49109f35c778080d3 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:35.966821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:35.989979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 193/382] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:36.312327 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2601fb0-1011-42d9-959d-8586387d115c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:36.444601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2601fb0-1011-42d9-959d-8586387d115c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:36.489179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd6eb0-53b5-48ef-be48-e4573a1f6379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:36.490210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd6eb0-53b5-48ef-be48-e4573a1f6379 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:36.490688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd6eb0-53b5-48ef-be48-e4573a1f6379 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:36.491151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd6eb0-53b5-48ef-be48-e4573a1f6379 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:36.505535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd6eb0-53b5-48ef-be48-e4573a1f6379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-c1dd6eb0-53b5-48ef-be48-e4573a1f6379 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:36.510756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:36.511651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:36.512469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:36.515597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 190/383] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:36.523580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e16e57-13ef-4c14-bcf8-0ee7afa15bd4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:36.619695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d2cef-4004-43eb-8965-896c4299235d None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:36.619984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d2cef-4004-43eb-8965-896c4299235d None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:36.620164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d2cef-4004-43eb-8965-896c4299235d None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:36.621627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 191/384] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:36.632689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b754987-a66a-4c54-8a59-9415878638fd None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:36.632964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b754987-a66a-4c54-8a59-9415878638fd None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:36.633230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b754987-a66a-4c54-8a59-9415878638fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:36.667969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-0b754987-a66a-4c54-8a59-9415878638fd None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:36.669175 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0b754987-a66a-4c54-8a59-9415878638fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:36.691410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:36.717966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e16e57-13ef-4c14-bcf8-0ee7afa15bd4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:36.769384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 192/385] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:36.770788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ccf99-e429-445f-8fae-70e9db4de5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:36.771414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ccf99-e429-445f-8fae-70e9db4de5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:36.771746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ccf99-e429-445f-8fae-70e9db4de5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:36.772018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ccf99-e429-445f-8fae-70e9db4de5c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:36.772690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ccf99-e429-445f-8fae-70e9db4de5c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:36.772982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ccf99-e429-445f-8fae-70e9db4de5c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:36.774343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ccf99-e429-445f-8fae-70e9db4de5c6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:36.789939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 194/386] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:36.797431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b754987-a66a-4c54-8a59-9415878638fd None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:36.920229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b754987-a66a-4c54-8a59-9415878638fd None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:36.957376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a36d6e-8ef1-45ce-b721-3bca7a25982a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:36.958011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a36d6e-8ef1-45ce-b721-3bca7a25982a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:36.958287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a36d6e-8ef1-45ce-b721-3bca7a25982a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:36.958553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a36d6e-8ef1-45ce-b721-3bca7a25982a None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:36.959674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a36d6e-8ef1-45ce-b721-3bca7a25982a None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:36.961101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a36d6e-8ef1-45ce-b721-3bca7a25982a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:36.963255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 193/387] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:36 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:36.968779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ccf99-e429-445f-8fae-70e9db4de5c6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:37.032638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:37.057651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207aec0-cdb5-43dd-9f7e-f23a328166d2 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:37.057909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207aec0-cdb5-43dd-9f7e-f23a328166d2 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:37.058178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207aec0-cdb5-43dd-9f7e-f23a328166d2 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:37.059764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 194/388] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:37.073095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14209c7f-6921-4f30-bc50-22f22df69e6e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:37.073366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14209c7f-6921-4f30-bc50-22f22df69e6e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:37.073638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14209c7f-6921-4f30-bc50-22f22df69e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:37.096781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ccf99-e429-445f-8fae-70e9db4de5c6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:37.111578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-14209c7f-6921-4f30-bc50-22f22df69e6e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:37.112778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-14209c7f-6921-4f30-bc50-22f22df69e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:37.125561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:37.146901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54670fd-c9e4-4375-86ba-6db97b0c6410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:37.147837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54670fd-c9e4-4375-86ba-6db97b0c6410 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:37.148255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54670fd-c9e4-4375-86ba-6db97b0c6410 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:37.148668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54670fd-c9e4-4375-86ba-6db97b0c6410 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:37.149984 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54670fd-c9e4-4375-86ba-6db97b0c6410 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:37.151631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54670fd-c9e4-4375-86ba-6db97b0c6410 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:37.156519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 195/389] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:36 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:37.163624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54670fd-c9e4-4375-86ba-6db97b0c6410 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:37.209843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 195/390] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:37.221230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14209c7f-6921-4f30-bc50-22f22df69e6e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:37.286929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54670fd-c9e4-4375-86ba-6db97b0c6410 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:37.305965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:37.326641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14209c7f-6921-4f30-bc50-22f22df69e6e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:37.327153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52801e68-e3e0-436a-bd41-e706ea0e977e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:37.327346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52801e68-e3e0-436a-bd41-e706ea0e977e None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:37.327649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52801e68-e3e0-436a-bd41-e706ea0e977e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:37.327924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52801e68-e3e0-436a-bd41-e706ea0e977e None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816/groups/ce8794bed1234e3e98a2099b19c9ce67/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:37.330162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52801e68-e3e0-436a-bd41-e706ea0e977e None admin] RBAC: Authorizing `identity:create_grant(project_id=04853db2b388437e916da478e79b0816, group_id=ce8794bed1234e3e98a2099b19c9ce67, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:37.331779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52801e68-e3e0-436a-bd41-e706ea0e977e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:37.367024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb856ce-6176-4f66-a688-617ac66830e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:37.368616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb856ce-6176-4f66-a688-617ac66830e3 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:37.368616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb856ce-6176-4f66-a688-617ac66830e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:37.368616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb856ce-6176-4f66-a688-617ac66830e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:37.379947 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:37.383857 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb856ce-6176-4f66-a688-617ac66830e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-4fb856ce-6176-4f66-a688-617ac66830e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:37.390539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:37.392993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:37.393796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 196/391] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:37.400404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb856ce-6176-4f66-a688-617ac66830e3 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:37.417849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 196/392] 10.222.0.62 () {58 vars in 1442 bytes} [Mon Sep 5 12:09:37 2022] PUT /identity/v3/projects/04853db2b388437e916da478e79b0816/groups/ce8794bed1234e3e98a2099b19c9ce67/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:09:37.554688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb856ce-6176-4f66-a688-617ac66830e3 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:37.607174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09923cc5-ceea-4eae-ad88-25ac1f45e82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:37.607826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09923cc5-ceea-4eae-ad88-25ac1f45e82d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:37.608105 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09923cc5-ceea-4eae-ad88-25ac1f45e82d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:37.608382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09923cc5-ceea-4eae-ad88-25ac1f45e82d None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:37.609055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09923cc5-ceea-4eae-ad88-25ac1f45e82d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:37.609354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09923cc5-ceea-4eae-ad88-25ac1f45e82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:37.610850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09923cc5-ceea-4eae-ad88-25ac1f45e82d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:37.620796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 197/393] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:37.626871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52801e68-e3e0-436a-bd41-e706ea0e977e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:37.684104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09923cc5-ceea-4eae-ad88-25ac1f45e82d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:37.751259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52801e68-e3e0-436a-bd41-e706ea0e977e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:37.758111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:37.790775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd0c29-7b4c-48a4-837f-8a9fbe2c100c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:37.791647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd0c29-7b4c-48a4-837f-8a9fbe2c100c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:37.791923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd0c29-7b4c-48a4-837f-8a9fbe2c100c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:37.792544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd0c29-7b4c-48a4-837f-8a9fbe2c100c None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:37.795224 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd0c29-7b4c-48a4-837f-8a9fbe2c100c None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:37.797351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd0c29-7b4c-48a4-837f-8a9fbe2c100c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:37.799674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 197/394] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:37 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:37.800535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09923cc5-ceea-4eae-ad88-25ac1f45e82d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:37.805230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd0c29-7b4c-48a4-837f-8a9fbe2c100c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:37.836746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f44b6-ac06-4204-a66e-d95a7b9825f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:37.837396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f44b6-ac06-4204-a66e-d95a7b9825f2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:37.837709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f44b6-ac06-4204-a66e-d95a7b9825f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:37.837985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f44b6-ac06-4204-a66e-d95a7b9825f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:37.849485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f44b6-ac06-4204-a66e-d95a7b9825f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-566f44b6-ac06-4204-a66e-d95a7b9825f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:37.856485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:37.857937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:37.858748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 198/395] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:37.864420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f44b6-ac06-4204-a66e-d95a7b9825f2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:37.868676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:37.944801 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd0c29-7b4c-48a4-837f-8a9fbe2c100c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:37.995316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de57493-36e4-46a2-ab98-946928172543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:37.996090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de57493-36e4-46a2-ab98-946928172543 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:37.996531 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de57493-36e4-46a2-ab98-946928172543 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:37.996946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de57493-36e4-46a2-ab98-946928172543 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:37.998181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f44b6-ac06-4204-a66e-d95a7b9825f2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:37.999133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de57493-36e4-46a2-ab98-946928172543 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:38.001165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de57493-36e4-46a2-ab98-946928172543 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:38.004310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 198/396] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:37 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:38.009529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de57493-36e4-46a2-ab98-946928172543 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:38.027028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:38.041579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f0d4d-5157-43d0-9270-0c855138fada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:38.042558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f0d4d-5157-43d0-9270-0c855138fada None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.043026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f0d4d-5157-43d0-9270-0c855138fada None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.043506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f0d4d-5157-43d0-9270-0c855138fada None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.044989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f0d4d-5157-43d0-9270-0c855138fada None admin] RBAC: Adding query filter params (name=member) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:38.045479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f0d4d-5157-43d0-9270-0c855138fada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:38.047050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f0d4d-5157-43d0-9270-0c855138fada None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:38.058127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 199/397] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:38.085604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f0d4d-5157-43d0-9270-0c855138fada None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:38.138206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de57493-36e4-46a2-ab98-946928172543 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:38.156514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:38.180018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22437119-671e-40f1-b0cc-1177bc089e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:38.180686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22437119-671e-40f1-b0cc-1177bc089e39 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.180963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22437119-671e-40f1-b0cc-1177bc089e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.181233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22437119-671e-40f1-b0cc-1177bc089e39 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816/groups/ce8794bed1234e3e98a2099b19c9ce67/roles/43008618c1dc4cd49109f35c778080d3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.183749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22437119-671e-40f1-b0cc-1177bc089e39 None admin] RBAC: Authorizing `identity:create_grant(project_id=04853db2b388437e916da478e79b0816, group_id=ce8794bed1234e3e98a2099b19c9ce67, role_id=43008618c1dc4cd49109f35c778080d3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:38.185667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22437119-671e-40f1-b0cc-1177bc089e39 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:38.249234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f0d4d-5157-43d0-9270-0c855138fada None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:38.274407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 199/398] 10.222.0.62 () {58 vars in 1442 bytes} [Mon Sep 5 12:09:38 2022] PUT /identity/v3/projects/04853db2b388437e916da478e79b0816/groups/ce8794bed1234e3e98a2099b19c9ce67/roles/43008618c1dc4cd49109f35c778080d3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:09:38.299116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e37688-a92c-451d-ae82-10f4c25e2a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:38.299781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e37688-a92c-451d-ae82-10f4c25e2a13 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.300146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e37688-a92c-451d-ae82-10f4c25e2a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.300528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e37688-a92c-451d-ae82-10f4c25e2a13 None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.301831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e37688-a92c-451d-ae82-10f4c25e2a13 None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:38.303352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e37688-a92c-451d-ae82-10f4c25e2a13 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:38.305421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 200/399] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:38 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:38.311083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22437119-671e-40f1-b0cc-1177bc089e39 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:38.458942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22437119-671e-40f1-b0cc-1177bc089e39 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:38.486550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511f36a-91a2-4ef1-a67d-1d8356ab4026 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.487050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511f36a-91a2-4ef1-a67d-1d8356ab4026 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.487127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511f36a-91a2-4ef1-a67d-1d8356ab4026 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.488675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 201/400] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:09:38.494222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5f72-d08d-4106-8fc3-ea9bb8394477 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.494791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5f72-d08d-4106-8fc3-ea9bb8394477 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.495266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5f72-d08d-4106-8fc3-ea9bb8394477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.503637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9dec4-8403-496b-a13e-e0de1811136e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:38.504334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9dec4-8403-496b-a13e-e0de1811136e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.504609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9dec4-8403-496b-a13e-e0de1811136e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.504878 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9dec4-8403-496b-a13e-e0de1811136e None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.506363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9dec4-8403-496b-a13e-e0de1811136e None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:38.507861 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9dec4-8403-496b-a13e-e0de1811136e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:38.510566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 200/401] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:38 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:38.516853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9dec4-8403-496b-a13e-e0de1811136e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:38.518937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8def5f72-d08d-4106-8fc3-ea9bb8394477 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:38.520316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8def5f72-d08d-4106-8fc3-ea9bb8394477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:38.571058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:38.577984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:38.611422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 202/402] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:09:38.668862 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9dec4-8403-496b-a13e-e0de1811136e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:38.713460 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0d415-2922-42a1-9ddf-7dadf87d0d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:38.714367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0d415-2922-42a1-9ddf-7dadf87d0d67 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.714792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0d415-2922-42a1-9ddf-7dadf87d0d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.715212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0d415-2922-42a1-9ddf-7dadf87d0d67 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/groups/ce8794bed1234e3e98a2099b19c9ce67/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.717919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0d415-2922-42a1-9ddf-7dadf87d0d67 None admin] RBAC: Authorizing `identity:create_grant(project_id=88bda6b61b9c4485a3366fe286e2c1d7, group_id=ce8794bed1234e3e98a2099b19c9ce67, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:38.719711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0d415-2922-42a1-9ddf-7dadf87d0d67 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:38.788554 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 201/403] 10.222.0.62 () {58 vars in 1442 bytes} [Mon Sep 5 12:09:38 2022] PUT /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/groups/ce8794bed1234e3e98a2099b19c9ce67/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:09:38.885742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd6efa-8b2e-492e-91b2-f21ea38f4e94 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.885926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd6efa-8b2e-492e-91b2-f21ea38f4e94 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.886194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd6efa-8b2e-492e-91b2-f21ea38f4e94 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.887620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 203/404] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:38.901327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab020c-ca26-40f1-aecb-d4e913d42457 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:38.901632 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab020c-ca26-40f1-aecb-d4e913d42457 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:38.901885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab020c-ca26-40f1-aecb-d4e913d42457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:38.929798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-49ab020c-ca26-40f1-aecb-d4e913d42457 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:38.931051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-49ab020c-ca26-40f1-aecb-d4e913d42457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:38.985473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd6efa-8b2e-492e-91b2-f21ea38f4e94 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.019963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 202/405] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:39.109215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd6efa-8b2e-492e-91b2-f21ea38f4e94 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:39.145982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685490-bd2d-4680-9df8-a3e79e54f64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:39.146549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685490-bd2d-4680-9df8-a3e79e54f64a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:39.146828 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685490-bd2d-4680-9df8-a3e79e54f64a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:39.147102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685490-bd2d-4680-9df8-a3e79e54f64a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:39.165917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b685490-bd2d-4680-9df8-a3e79e54f64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-3b685490-bd2d-4680-9df8-a3e79e54f64a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:39.171084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:39.172003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:39.172820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 204/406] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:39.180632 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab020c-ca26-40f1-aecb-d4e913d42457 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.230012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:39.316777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab020c-ca26-40f1-aecb-d4e913d42457 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:39.352664 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c32680-004f-496d-ada8-6a07ef830c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:39.353280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c32680-004f-496d-ada8-6a07ef830c8e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:39.353557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c32680-004f-496d-ada8-6a07ef830c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:39.353832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c32680-004f-496d-ada8-6a07ef830c8e None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:39.354500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c32680-004f-496d-ada8-6a07ef830c8e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:39.354798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c32680-004f-496d-ada8-6a07ef830c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:39.356248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c32680-004f-496d-ada8-6a07ef830c8e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:39.365046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 203/407] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:39.370750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685490-bd2d-4680-9df8-a3e79e54f64a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.409161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c32680-004f-496d-ada8-6a07ef830c8e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.430075 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:39.507782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685490-bd2d-4680-9df8-a3e79e54f64a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:39.539884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:39.551988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c32680-004f-496d-ada8-6a07ef830c8e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:39.585627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757aa480-6b72-4313-b843-50c6177a7790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:39.586874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aa480-6b72-4313-b843-50c6177a7790 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:39.587382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aa480-6b72-4313-b843-50c6177a7790 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:39.587780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aa480-6b72-4313-b843-50c6177a7790 None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:39.590857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117998db-116f-4204-b0c0-ac58d9f7906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:39.592230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757aa480-6b72-4313-b843-50c6177a7790 None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:39.592536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117998db-116f-4204-b0c0-ac58d9f7906a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:39.592949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117998db-116f-4204-b0c0-ac58d9f7906a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:39.593416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117998db-116f-4204-b0c0-ac58d9f7906a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:39.594176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757aa480-6b72-4313-b843-50c6177a7790 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:39.595963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 205/408] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:39 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:39.601419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757aa480-6b72-4313-b843-50c6177a7790 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.609129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117998db-116f-4204-b0c0-ac58d9f7906a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-117998db-116f-4204-b0c0-ac58d9f7906a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:39.621540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:39.622497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:39.623323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 204/409] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:39.629666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117998db-116f-4204-b0c0-ac58d9f7906a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.750304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117998db-116f-4204-b0c0-ac58d9f7906a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:39.787216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d5be2-2435-46c3-95e1-f680fcfeb9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:39.788198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d5be2-2435-46c3-95e1-f680fcfeb9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:39.788680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d5be2-2435-46c3-95e1-f680fcfeb9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:39.789170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d5be2-2435-46c3-95e1-f680fcfeb9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:39.790046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d5be2-2435-46c3-95e1-f680fcfeb9a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:39.790568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d5be2-2435-46c3-95e1-f680fcfeb9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:39.792166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d5be2-2435-46c3-95e1-f680fcfeb9a9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:39.802253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 205/410] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:39.810542 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d5be2-2435-46c3-95e1-f680fcfeb9a9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.834137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:39.846309 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757aa480-6b72-4313-b843-50c6177a7790 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:39.867171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:39.931291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d5be2-2435-46c3-95e1-f680fcfeb9a9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:39.948267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22798475-8e2f-4310-9394-beaa8457600f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:39.949205 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22798475-8e2f-4310-9394-beaa8457600f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:39.949205 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22798475-8e2f-4310-9394-beaa8457600f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:39.949489 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22798475-8e2f-4310-9394-beaa8457600f None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:39.950653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22798475-8e2f-4310-9394-beaa8457600f None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:39.952152 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22798475-8e2f-4310-9394-beaa8457600f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:39.953784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 206/411] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:39 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:39.959373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22798475-8e2f-4310-9394-beaa8457600f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:39.980701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ed691-fc5c-4488-9b02-1206fdd321bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:39.983152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ed691-fc5c-4488-9b02-1206fdd321bc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:39.983887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ed691-fc5c-4488-9b02-1206fdd321bc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:39.984749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ed691-fc5c-4488-9b02-1206fdd321bc None admin] PATH_INFO: `/v3/groups/5689a76b675d494baa09a9faf3795f04` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:39.986911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ed691-fc5c-4488-9b02-1206fdd321bc None admin] RBAC: Authorizing `identity:get_group(group_id=5689a76b675d494baa09a9faf3795f04)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:39.989444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ed691-fc5c-4488-9b02-1206fdd321bc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:39.992716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 206/412] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:39 2022] GET /identity/v3/groups/5689a76b675d494baa09a9faf3795f04 => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:39.998980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ed691-fc5c-4488-9b02-1206fdd321bc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:40.137741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:40.141590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22798475-8e2f-4310-9394-beaa8457600f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:40.148084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ed691-fc5c-4488-9b02-1206fdd321bc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:40.214436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cc0fa-d99f-4c3b-b992-7520d1ebf4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:40.215085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cc0fa-d99f-4c3b-b992-7520d1ebf4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:40.216042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d8777-3bb6-4f24-8954-3fd0ae6fd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:40.217123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cc0fa-d99f-4c3b-b992-7520d1ebf4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:40.217410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cc0fa-d99f-4c3b-b992-7520d1ebf4c7 None admin] PATH_INFO: `/v3/projects/d80f282fe9da41e0b08a14ba58840408` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:40.218252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8777-3bb6-4f24-8954-3fd0ae6fd671 None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:40.218538 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8777-3bb6-4f24-8954-3fd0ae6fd671 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:40.218809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8777-3bb6-4f24-8954-3fd0ae6fd671 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/groups/ce8794bed1234e3e98a2099b19c9ce67/roles/43008618c1dc4cd49109f35c778080d3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:40.220020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cc0fa-d99f-4c3b-b992-7520d1ebf4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=d80f282fe9da41e0b08a14ba58840408)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:40.221456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cc0fa-d99f-4c3b-b992-7520d1ebf4c7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:40.223115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 207/413] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:39 2022] GET /identity/v3/projects/d80f282fe9da41e0b08a14ba58840408 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:40.224893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d8777-3bb6-4f24-8954-3fd0ae6fd671 None admin] RBAC: Authorizing `identity:create_grant(project_id=88bda6b61b9c4485a3366fe286e2c1d7, group_id=ce8794bed1234e3e98a2099b19c9ce67, role_id=43008618c1dc4cd49109f35c778080d3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:40.226398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d8777-3bb6-4f24-8954-3fd0ae6fd671 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:40.228771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cc0fa-d99f-4c3b-b992-7520d1ebf4c7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:40.294684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 207/414] 10.222.0.62 () {58 vars in 1442 bytes} [Mon Sep 5 12:09:39 2022] PUT /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7/groups/ce8794bed1234e3e98a2099b19c9ce67/roles/43008618c1dc4cd49109f35c778080d3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:40.296383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bfafd-a25c-4062-8363-b63c421dc20a None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:40.296679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bfafd-a25c-4062-8363-b63c421dc20a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:40.298525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bfafd-a25c-4062-8363-b63c421dc20a None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:40.301849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 208/415] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:40.310784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6b79d-f661-401a-8b41-a4ebbc88df4a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:40.311375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6b79d-f661-401a-8b41-a4ebbc88df4a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:40.311870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6b79d-f661-401a-8b41-a4ebbc88df4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:40.349438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-2ab6b79d-f661-401a-8b41-a4ebbc88df4a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:40.351025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2ab6b79d-f661-401a-8b41-a4ebbc88df4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:40.371251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cc0fa-d99f-4c3b-b992-7520d1ebf4c7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:40.420273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5312c0-c940-4da5-bde1-90adf6c2817b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:40.421233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5312c0-c940-4da5-bde1-90adf6c2817b None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:40.421677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5312c0-c940-4da5-bde1-90adf6c2817b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:40.422097 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5312c0-c940-4da5-bde1-90adf6c2817b None admin] PATH_INFO: `/v3/projects/d80f282fe9da41e0b08a14ba58840408/groups/5689a76b675d494baa09a9faf3795f04/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:40.426682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5312c0-c940-4da5-bde1-90adf6c2817b None admin] RBAC: Authorizing `identity:create_grant(project_id=d80f282fe9da41e0b08a14ba58840408, group_id=5689a76b675d494baa09a9faf3795f04, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:40.430990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5312c0-c940-4da5-bde1-90adf6c2817b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:40.440728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:40.485205 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 209/416] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:40.516853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 208/417] 10.222.0.62 () {58 vars in 1442 bytes} [Mon Sep 5 12:09:40 2022] PUT /identity/v3/projects/d80f282fe9da41e0b08a14ba58840408/groups/5689a76b675d494baa09a9faf3795f04/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:09:40.771832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6b79d-f661-401a-8b41-a4ebbc88df4a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:40.809645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1fe51-859c-4f5c-a145-c5823e253c7b None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:40.810703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1fe51-859c-4f5c-a145-c5823e253c7b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:40.810778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1fe51-859c-4f5c-a145-c5823e253c7b None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:40.812414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 209/418] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:09:40.820899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c4203-2d5a-459f-a5ff-93c7374cc51d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:40.821808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c4203-2d5a-459f-a5ff-93c7374cc51d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:40.822283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c4203-2d5a-459f-a5ff-93c7374cc51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:40.848181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c54c4203-2d5a-459f-a5ff-93c7374cc51d None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:40.849836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c54c4203-2d5a-459f-a5ff-93c7374cc51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:40.863897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:40.929631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 210/419] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:40.946515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6b79d-f661-401a-8b41-a4ebbc88df4a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:40.983914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5ae59-f0e2-471d-b844-cbe47a75a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:40.984556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5ae59-f0e2-471d-b844-cbe47a75a359 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:40.984841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5ae59-f0e2-471d-b844-cbe47a75a359 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:40.985118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5ae59-f0e2-471d-b844-cbe47a75a359 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:40.995967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5ae59-f0e2-471d-b844-cbe47a75a359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-f7c5ae59-f0e2-471d-b844-cbe47a75a359 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:41.001753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:41.002638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:41.003449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 210/420] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:41.008589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c4203-2d5a-459f-a5ff-93c7374cc51d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:41.146995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c4203-2d5a-459f-a5ff-93c7374cc51d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:41.194395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3d3e4-928d-423b-a96b-44463b725f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:41.195021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3d3e4-928d-423b-a96b-44463b725f70 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:41.195303 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3d3e4-928d-423b-a96b-44463b725f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:41.195596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3d3e4-928d-423b-a96b-44463b725f70 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:41.196271 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3d3e4-928d-423b-a96b-44463b725f70 None admin] RBAC: Adding query filter params (name=member) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:41.196575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3d3e4-928d-423b-a96b-44463b725f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:41.197950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3d3e4-928d-423b-a96b-44463b725f70 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:41.211905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 211/421] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:41.213324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5ae59-f0e2-471d-b844-cbe47a75a359 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:41.240736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3d3e4-928d-423b-a96b-44463b725f70 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:41.244742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:41.245479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:41.339638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5ae59-f0e2-471d-b844-cbe47a75a359 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:41.351203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3d3e4-928d-423b-a96b-44463b725f70 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:41.378575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b305fe-0fef-45c3-a9dd-e88c6134cb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:41.379419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b305fe-0fef-45c3-a9dd-e88c6134cb26 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:41.379865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b305fe-0fef-45c3-a9dd-e88c6134cb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:41.380396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b305fe-0fef-45c3-a9dd-e88c6134cb26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:41.397549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0d54b-8736-43b3-8fcc-2a6a9dcfd930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:41.398181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0d54b-8736-43b3-8fcc-2a6a9dcfd930 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:41.398464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0d54b-8736-43b3-8fcc-2a6a9dcfd930 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:41.401382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0d54b-8736-43b3-8fcc-2a6a9dcfd930 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:41.416439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0d54b-8736-43b3-8fcc-2a6a9dcfd930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-49f0d54b-8736-43b3-8fcc-2a6a9dcfd930 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:41.424030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:41.425128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:41.426079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 212/422] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:41.426740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3082a4-4592-4e74-a604-fe0e235ec087 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:41.431520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3082a4-4592-4e74-a604-fe0e235ec087 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:41.431520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3082a4-4592-4e74-a604-fe0e235ec087 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:41.431520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 213/423] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:41.434748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3082a4-4592-4e74-a604-fe0e235ec087 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:41.472351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b305fe-0fef-45c3-a9dd-e88c6134cb26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:41.474415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b305fe-0fef-45c3-a9dd-e88c6134cb26 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:41.507875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:41.512207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:41.555918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3082a4-4592-4e74-a604-fe0e235ec087 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:41.590423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 211/424] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:41.592460 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964be32-3d71-47f8-95d6-d43d092f383c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:41.592753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964be32-3d71-47f8-95d6-d43d092f383c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:41.593033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964be32-3d71-47f8-95d6-d43d092f383c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:41.596298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dacb84-6def-4377-9fb3-febb408790de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:41.597597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dacb84-6def-4377-9fb3-febb408790de None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:41.598321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dacb84-6def-4377-9fb3-febb408790de None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:41.599017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dacb84-6def-4377-9fb3-febb408790de None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:41.600197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dacb84-6def-4377-9fb3-febb408790de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:41.600789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dacb84-6def-4377-9fb3-febb408790de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:41.602339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dacb84-6def-4377-9fb3-febb408790de None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:41.611760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6964be32-3d71-47f8-95d6-d43d092f383c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:41.612404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 214/425] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:41.617357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6964be32-3d71-47f8-95d6-d43d092f383c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:41.618212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dacb84-6def-4377-9fb3-febb408790de None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:41.691208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 212/426] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:41.691208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964be32-3d71-47f8-95d6-d43d092f383c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:41.752597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:41.763123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dacb84-6def-4377-9fb3-febb408790de None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:41.798318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964be32-3d71-47f8-95d6-d43d092f383c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:41.814046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:41.833376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a0d00-5847-4013-8a2c-7fa6fdfccb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:41.833999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a0d00-5847-4013-8a2c-7fa6fdfccb80 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:41.834131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c568233-3243-431d-9e45-a04d5d502948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:41.834321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a0d00-5847-4013-8a2c-7fa6fdfccb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:41.834544 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a0d00-5847-4013-8a2c-7fa6fdfccb80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:41.835991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c568233-3243-431d-9e45-a04d5d502948 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:41.836266 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c568233-3243-431d-9e45-a04d5d502948 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:41.836821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c568233-3243-431d-9e45-a04d5d502948 None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:41.838561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c568233-3243-431d-9e45-a04d5d502948 None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:41.840052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c568233-3243-431d-9e45-a04d5d502948 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:41.845457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 215/427] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:41 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:41.853031 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c568233-3243-431d-9e45-a04d5d502948 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:41.912847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a0d00-5847-4013-8a2c-7fa6fdfccb80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:41.914321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a0d00-5847-4013-8a2c-7fa6fdfccb80 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:41.976653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:42.022705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 213/428] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:42.032514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a0d00-5847-4013-8a2c-7fa6fdfccb80 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:42.075689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c568233-3243-431d-9e45-a04d5d502948 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:42.118628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265db69-78ad-487b-b3f1-43bdebb1f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:42.119172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265db69-78ad-487b-b3f1-43bdebb1f73b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.119449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265db69-78ad-487b-b3f1-43bdebb1f73b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.119773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265db69-78ad-487b-b3f1-43bdebb1f73b None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.120879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265db69-78ad-487b-b3f1-43bdebb1f73b None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:42.122472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265db69-78ad-487b-b3f1-43bdebb1f73b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:42.124957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 216/429] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:41 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:42.125711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265db69-78ad-487b-b3f1-43bdebb1f73b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:42.146812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a0d00-5847-4013-8a2c-7fa6fdfccb80 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:42.186527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99ddb0-da7e-4c1e-9672-f435e5df042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:42.187172 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99ddb0-da7e-4c1e-9672-f435e5df042b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.187458 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99ddb0-da7e-4c1e-9672-f435e5df042b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.188025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99ddb0-da7e-4c1e-9672-f435e5df042b None admin] PATH_INFO: `/v3/projects/04853db2b388437e916da478e79b0816` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.189603 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99ddb0-da7e-4c1e-9672-f435e5df042b None admin] RBAC: Authorizing `identity:get_project(project_id=04853db2b388437e916da478e79b0816)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:42.191615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99ddb0-da7e-4c1e-9672-f435e5df042b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:42.193976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 214/430] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:42 2022] GET /identity/v3/projects/04853db2b388437e916da478e79b0816 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:42.194823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99ddb0-da7e-4c1e-9672-f435e5df042b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:42.206198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:42.232778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:42.251521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265db69-78ad-487b-b3f1-43bdebb1f73b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:42.296155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9bf23-34dc-451f-9228-db632e60dff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:42.297292 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9bf23-34dc-451f-9228-db632e60dff0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.297714 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9bf23-34dc-451f-9228-db632e60dff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.298360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9bf23-34dc-451f-9228-db632e60dff0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.313322 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9bf23-34dc-451f-9228-db632e60dff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:42.317376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99ddb0-da7e-4c1e-9672-f435e5df042b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-26f9bf23-34dc-451f-9228-db632e60dff0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:42.322624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:42.323556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:42.324376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 217/431] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:42.325053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9bf23-34dc-451f-9228-db632e60dff0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:42.358543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dcc5c-259a-4e0a-9c23-e1aeb3b3122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:42.359503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dcc5c-259a-4e0a-9c23-e1aeb3b3122f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.360024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dcc5c-259a-4e0a-9c23-e1aeb3b3122f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.360490 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dcc5c-259a-4e0a-9c23-e1aeb3b3122f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.361425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dcc5c-259a-4e0a-9c23-e1aeb3b3122f None admin] RBAC: Adding query filter params (group.id=ce8794bed1234e3e98a2099b19c9ce67, role.id=0062155b0fbf41a19c9110ad71a046e3, scope.project.id=04853db2b388437e916da478e79b0816) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:42.361927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dcc5c-259a-4e0a-9c23-e1aeb3b3122f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:42.363728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dcc5c-259a-4e0a-9c23-e1aeb3b3122f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:42.370187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 215/432] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Sep 5 12:09:42 2022] GET /identity/v3/role_assignments?group.id=ce8794bed1234e3e98a2099b19c9ce67&scope.project.id=04853db2b388437e916da478e79b0816&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:42.371229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dcc5c-259a-4e0a-9c23-e1aeb3b3122f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:42.439081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9bf23-34dc-451f-9228-db632e60dff0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:42.475704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:42.484492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:42.488451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dcc5c-259a-4e0a-9c23-e1aeb3b3122f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:42.513487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c44f-debb-4e5b-8229-42361adefb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:42.514147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c44f-debb-4e5b-8229-42361adefb5e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.514424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c44f-debb-4e5b-8229-42361adefb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.514696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c44f-debb-4e5b-8229-42361adefb5e None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.516447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6c44f-debb-4e5b-8229-42361adefb5e None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:42.517636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6c44f-debb-4e5b-8229-42361adefb5e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:42.525603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 218/433] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:42 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:42.526708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c44f-debb-4e5b-8229-42361adefb5e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:42.534158 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ba5b8-a92a-4cc1-b0a2-c8313130b0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:42.535121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ba5b8-a92a-4cc1-b0a2-c8313130b0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.535668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ba5b8-a92a-4cc1-b0a2-c8313130b0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.536242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ba5b8-a92a-4cc1-b0a2-c8313130b0a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.537104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ba5b8-a92a-4cc1-b0a2-c8313130b0a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:42.537599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ba5b8-a92a-4cc1-b0a2-c8313130b0a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:42.539178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ba5b8-a92a-4cc1-b0a2-c8313130b0a9 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:42.549299 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 216/434] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:09:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:42.561278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ba5b8-a92a-4cc1-b0a2-c8313130b0a9 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:42.665329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c44f-debb-4e5b-8229-42361adefb5e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:42.681925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ba5b8-a92a-4cc1-b0a2-c8313130b0a9 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:42.704661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79eea30-b49c-4805-a5b6-013ebf2c51f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:42.705391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79eea30-b49c-4805-a5b6-013ebf2c51f8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.705706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79eea30-b49c-4805-a5b6-013ebf2c51f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.706053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79eea30-b49c-4805-a5b6-013ebf2c51f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.706793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79eea30-b49c-4805-a5b6-013ebf2c51f8 None admin] RBAC: Adding query filter params (group.id=ce8794bed1234e3e98a2099b19c9ce67, role.id=43008618c1dc4cd49109f35c778080d3, scope.project.id=04853db2b388437e916da478e79b0816) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:42.707104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79eea30-b49c-4805-a5b6-013ebf2c51f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:42.708809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79eea30-b49c-4805-a5b6-013ebf2c51f8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:42.718449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 219/435] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Sep 5 12:09:42 2022] GET /identity/v3/role_assignments?group.id=ce8794bed1234e3e98a2099b19c9ce67&scope.project.id=04853db2b388437e916da478e79b0816&role.id=43008618c1dc4cd49109f35c778080d3 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:42.720264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aca796-b0f4-445a-9874-44472c9d4da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:42.720896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aca796-b0f4-445a-9874-44472c9d4da1 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.721210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aca796-b0f4-445a-9874-44472c9d4da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.721503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aca796-b0f4-445a-9874-44472c9d4da1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.752082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:42.784073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e2145-703d-47c9-ba23-51a94577fa5e None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.784436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e2145-703d-47c9-ba23-51a94577fa5e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.784608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e2145-703d-47c9-ba23-51a94577fa5e None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.787656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 220/436] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:42.793804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de4d2a-065f-4133-9d09-acabc5aea076 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.794257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de4d2a-065f-4133-9d09-acabc5aea076 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.794583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de4d2a-065f-4133-9d09-acabc5aea076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.806274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aca796-b0f4-445a-9874-44472c9d4da1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:42.807829 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aca796-b0f4-445a-9874-44472c9d4da1 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:42.815071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e8de4d2a-065f-4133-9d09-acabc5aea076 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:42.816313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e8de4d2a-065f-4133-9d09-acabc5aea076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:42.882231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:42.905556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 221/437] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:42.906839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98903052-a1ce-4181-b3ae-7c3f2ea3f9b6 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.907406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98903052-a1ce-4181-b3ae-7c3f2ea3f9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.907406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98903052-a1ce-4181-b3ae-7c3f2ea3f9b6 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.909780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 222/438] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:42.916832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152ac43-0186-4124-bd0d-ea757a8bc850 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:42.917119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152ac43-0186-4124-bd0d-ea757a8bc850 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:42.917460 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152ac43-0186-4124-bd0d-ea757a8bc850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:42.933542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 217/439] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:42.936148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c152ac43-0186-4124-bd0d-ea757a8bc850 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:42.937342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c152ac43-0186-4124-bd0d-ea757a8bc850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:42.939471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aca796-b0f4-445a-9874-44472c9d4da1 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:43.012601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:43.019067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 223/440] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:43.085186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aca796-b0f4-445a-9874-44472c9d4da1 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:43.126550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9d2fb-c2bd-417c-a70c-c40e4711d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:43.127417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9d2fb-c2bd-417c-a70c-c40e4711d2bb None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:43.127881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9d2fb-c2bd-417c-a70c-c40e4711d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:43.128318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9d2fb-c2bd-417c-a70c-c40e4711d2bb None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:43.129595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9d2fb-c2bd-417c-a70c-c40e4711d2bb None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:43.131182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9d2fb-c2bd-417c-a70c-c40e4711d2bb None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:43.133345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 218/441] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:42 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:43.138871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152ac43-0186-4124-bd0d-ea757a8bc850 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:43.225025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9d2fb-c2bd-417c-a70c-c40e4711d2bb None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:43.260551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152ac43-0186-4124-bd0d-ea757a8bc850 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:43.277897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:43.302236 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a42732-2273-4e8f-941c-9e0c5a8fc413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:43.303473 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42732-2273-4e8f-941c-9e0c5a8fc413 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:43.304277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42732-2273-4e8f-941c-9e0c5a8fc413 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:43.304807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42732-2273-4e8f-941c-9e0c5a8fc413 None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:43.306040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a42732-2273-4e8f-941c-9e0c5a8fc413 None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:43.307470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a42732-2273-4e8f-941c-9e0c5a8fc413 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:43.309113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 224/442] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:43 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:43.312402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a42732-2273-4e8f-941c-9e0c5a8fc413 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:43.350231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9d2fb-c2bd-417c-a70c-c40e4711d2bb None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:43.370576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:43.384596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d00916-2407-4cd5-be2b-088138b5c070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:43.385238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d00916-2407-4cd5-be2b-088138b5c070 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:43.385513 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d00916-2407-4cd5-be2b-088138b5c070 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:43.385788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d00916-2407-4cd5-be2b-088138b5c070 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:43.396423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d00916-2407-4cd5-be2b-088138b5c070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-c0d00916-2407-4cd5-be2b-088138b5c070 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:43.401816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:43.402779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 219/443] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:43.403678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d00916-2407-4cd5-be2b-088138b5c070 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:43.411134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a42732-2273-4e8f-941c-9e0c5a8fc413 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:43.446826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b8e88-e706-434e-b12c-0a1e3641e2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:43.447452 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b8e88-e706-434e-b12c-0a1e3641e2bb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:43.447757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b8e88-e706-434e-b12c-0a1e3641e2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:43.448029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b8e88-e706-434e-b12c-0a1e3641e2bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:43.458414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b8e88-e706-434e-b12c-0a1e3641e2bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-376b8e88-e706-434e-b12c-0a1e3641e2bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:43.463522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:43.464607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:43.465494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 225/444] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:43.466912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b8e88-e706-434e-b12c-0a1e3641e2bb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:43.504458 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d00916-2407-4cd5-be2b-088138b5c070 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:43.508614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:43.537902 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8f76f-a300-4e63-8297-4783b7288a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:43.538495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8f76f-a300-4e63-8297-4783b7288a31 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:43.538773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8f76f-a300-4e63-8297-4783b7288a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:43.539045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8f76f-a300-4e63-8297-4783b7288a31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:43.539793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8f76f-a300-4e63-8297-4783b7288a31 None admin] RBAC: Adding query filter params (group.id=ce8794bed1234e3e98a2099b19c9ce67, role.id=0062155b0fbf41a19c9110ad71a046e3, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:43.540100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8f76f-a300-4e63-8297-4783b7288a31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:43.541529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8f76f-a300-4e63-8297-4783b7288a31 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:43.547988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 220/445] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Sep 5 12:09:43 2022] GET /identity/v3/role_assignments?group.id=ce8794bed1234e3e98a2099b19c9ce67&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=0062155b0fbf41a19c9110ad71a046e3 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:43.548747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8f76f-a300-4e63-8297-4783b7288a31 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:43.565999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b8e88-e706-434e-b12c-0a1e3641e2bb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:43.606158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6052370-2aa8-4d13-9eab-137bc15b25d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:43.607741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6052370-2aa8-4d13-9eab-137bc15b25d0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:43.608030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6052370-2aa8-4d13-9eab-137bc15b25d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:43.608304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6052370-2aa8-4d13-9eab-137bc15b25d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:43.608991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6052370-2aa8-4d13-9eab-137bc15b25d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:43.609333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6052370-2aa8-4d13-9eab-137bc15b25d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:43.610760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6052370-2aa8-4d13-9eab-137bc15b25d0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:43.613938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:43.624341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 226/446] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:43.631295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6052370-2aa8-4d13-9eab-137bc15b25d0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:43.679527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8f76f-a300-4e63-8297-4783b7288a31 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:43.722471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339858d0-4041-49b3-8c33-d06cda607247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:43.723083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339858d0-4041-49b3-8c33-d06cda607247 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:43.723362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339858d0-4041-49b3-8c33-d06cda607247 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:43.723666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339858d0-4041-49b3-8c33-d06cda607247 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:43.724324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339858d0-4041-49b3-8c33-d06cda607247 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:43.726322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339858d0-4041-49b3-8c33-d06cda607247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:43.726322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339858d0-4041-49b3-8c33-d06cda607247 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:43.735130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 221/447] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:43.741368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339858d0-4041-49b3-8c33-d06cda607247 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:43.745287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6052370-2aa8-4d13-9eab-137bc15b25d0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:43.748016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:43.782565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eadfa1-1a55-4f32-9387-d3b740195067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:43.783503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eadfa1-1a55-4f32-9387-d3b740195067 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:43.783970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eadfa1-1a55-4f32-9387-d3b740195067 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:43.784249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eadfa1-1a55-4f32-9387-d3b740195067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:43.853262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339858d0-4041-49b3-8c33-d06cda607247 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:43.862216 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:43.866869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eadfa1-1a55-4f32-9387-d3b740195067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:43.868381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eadfa1-1a55-4f32-9387-d3b740195067 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:43.894702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4662da-dbfd-44b7-95a7-ac076f98c493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:43.895319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4662da-dbfd-44b7-95a7-ac076f98c493 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:43.895637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4662da-dbfd-44b7-95a7-ac076f98c493 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:43.895910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4662da-dbfd-44b7-95a7-ac076f98c493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:43.964193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4662da-dbfd-44b7-95a7-ac076f98c493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:43.965646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4662da-dbfd-44b7-95a7-ac076f98c493 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:43.973710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 227/448] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 12:09:43.979313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eadfa1-1a55-4f32-9387-d3b740195067 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:43.982102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:44.070814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 222/449] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:44.076947 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4662da-dbfd-44b7-95a7-ac076f98c493 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:44.078314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eadfa1-1a55-4f32-9387-d3b740195067 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:44.084391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:44.112031 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22b897-81e7-4f3f-8286-71a08b07b921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:44.112645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22b897-81e7-4f3f-8286-71a08b07b921 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:44.112919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22b897-81e7-4f3f-8286-71a08b07b921 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:44.113242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22b897-81e7-4f3f-8286-71a08b07b921 None admin] PATH_INFO: `/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:44.114347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22b897-81e7-4f3f-8286-71a08b07b921 None admin] RBAC: Authorizing `identity:get_project(project_id=88bda6b61b9c4485a3366fe286e2c1d7)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:44.115788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22b897-81e7-4f3f-8286-71a08b07b921 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:44.117372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 228/450] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:43 2022] GET /identity/v3/projects/88bda6b61b9c4485a3366fe286e2c1d7 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:44.123212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22b897-81e7-4f3f-8286-71a08b07b921 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:44.178036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4662da-dbfd-44b7-95a7-ac076f98c493 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:44.191689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:44.213882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2bc95-1e10-486b-8e13-40f0e77ac3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:44.214503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2bc95-1e10-486b-8e13-40f0e77ac3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:44.214784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2bc95-1e10-486b-8e13-40f0e77ac3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:44.215056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2bc95-1e10-486b-8e13-40f0e77ac3c7 None admin] PATH_INFO: `/v3/projects/d80f282fe9da41e0b08a14ba58840408` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:44.216178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2bc95-1e10-486b-8e13-40f0e77ac3c7 None admin] RBAC: Authorizing `identity:get_project(project_id=d80f282fe9da41e0b08a14ba58840408)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:44.217654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2bc95-1e10-486b-8e13-40f0e77ac3c7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:44.219644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 223/451] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:09:44 2022] GET /identity/v3/projects/d80f282fe9da41e0b08a14ba58840408 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:44.222288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22b897-81e7-4f3f-8286-71a08b07b921 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:44.224846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2bc95-1e10-486b-8e13-40f0e77ac3c7 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:44.256001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f258b-bd2e-4233-a493-d062cbfeb20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:44.256614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f258b-bd2e-4233-a493-d062cbfeb20e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:44.256887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f258b-bd2e-4233-a493-d062cbfeb20e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:44.257155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f258b-bd2e-4233-a493-d062cbfeb20e None admin] PATH_INFO: `/v3/groups/ce8794bed1234e3e98a2099b19c9ce67` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:44.258256 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f258b-bd2e-4233-a493-d062cbfeb20e None admin] RBAC: Authorizing `identity:get_group(group_id=ce8794bed1234e3e98a2099b19c9ce67)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:44.259702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f258b-bd2e-4233-a493-d062cbfeb20e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:44.261243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 229/452] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:44 2022] GET /identity/v3/groups/ce8794bed1234e3e98a2099b19c9ce67 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:44.267445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f258b-bd2e-4233-a493-d062cbfeb20e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:44.298850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:44.323579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2bc95-1e10-486b-8e13-40f0e77ac3c7 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:44.356890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965803d1-e606-43f2-9a60-b9185115414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:44.357544 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965803d1-e606-43f2-9a60-b9185115414d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:44.357827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965803d1-e606-43f2-9a60-b9185115414d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:44.358101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965803d1-e606-43f2-9a60-b9185115414d None admin] PATH_INFO: `/v3/groups/5689a76b675d494baa09a9faf3795f04` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:44.359221 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965803d1-e606-43f2-9a60-b9185115414d None admin] RBAC: Authorizing `identity:get_group(group_id=5689a76b675d494baa09a9faf3795f04)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:44.360702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965803d1-e606-43f2-9a60-b9185115414d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:44.362259 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 224/453] 10.222.0.62 () {56 vars in 1180 bytes} [Mon Sep 5 12:09:44 2022] GET /identity/v3/groups/5689a76b675d494baa09a9faf3795f04 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:44.363855 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f258b-bd2e-4233-a493-d062cbfeb20e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:44.368000 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965803d1-e606-43f2-9a60-b9185115414d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:44.397886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c964f-73c6-405d-9b2d-2a87adbdbb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:44.401409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c964f-73c6-405d-9b2d-2a87adbdbb55 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:44.401409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c964f-73c6-405d-9b2d-2a87adbdbb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:44.401409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c964f-73c6-405d-9b2d-2a87adbdbb55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:44.401409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c964f-73c6-405d-9b2d-2a87adbdbb55 None admin] RBAC: Adding query filter params (group.id=ce8794bed1234e3e98a2099b19c9ce67, role.id=43008618c1dc4cd49109f35c778080d3, scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:44.401409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c964f-73c6-405d-9b2d-2a87adbdbb55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:44.401823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c964f-73c6-405d-9b2d-2a87adbdbb55 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:44.407309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 230/454] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Sep 5 12:09:44 2022] GET /identity/v3/role_assignments?group.id=ce8794bed1234e3e98a2099b19c9ce67&scope.project.id=88bda6b61b9c4485a3366fe286e2c1d7&role.id=43008618c1dc4cd49109f35c778080d3 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:44.475101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965803d1-e606-43f2-9a60-b9185115414d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:44.511031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee12fe-4c3d-4498-afe3-bbde3a325d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:44.511681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee12fe-4c3d-4498-afe3-bbde3a325d71 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:44.511961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee12fe-4c3d-4498-afe3-bbde3a325d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:44.512231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee12fe-4c3d-4498-afe3-bbde3a325d71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:44.512971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee12fe-4c3d-4498-afe3-bbde3a325d71 None admin] RBAC: Adding query filter params (group.id=5689a76b675d494baa09a9faf3795f04, role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=d80f282fe9da41e0b08a14ba58840408) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:44.513314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee12fe-4c3d-4498-afe3-bbde3a325d71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:44.514776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee12fe-4c3d-4498-afe3-bbde3a325d71 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:44.521179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 225/455] 10.222.0.62 () {56 vars in 1376 bytes} [Mon Sep 5 12:09:44 2022] GET /identity/v3/role_assignments?group.id=5689a76b675d494baa09a9faf3795f04&scope.project.id=d80f282fe9da41e0b08a14ba58840408&role.id=705907b9bdec47cebf88b868a6a56356 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:46.823417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39399196-0cc8-406a-807b-d8c86a6d1e77 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:46.823978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39399196-0cc8-406a-807b-d8c86a6d1e77 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:46.823978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39399196-0cc8-406a-807b-d8c86a6d1e77 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:46.825591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 231/456] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:46.831500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c25c07-848e-4ead-9650-406b20d7b364 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:46.831936 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c25c07-848e-4ead-9650-406b20d7b364 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:46.832121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c25c07-848e-4ead-9650-406b20d7b364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:46.846800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:46.857019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-89c25c07-848e-4ead-9650-406b20d7b364 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:46.861349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-89c25c07-848e-4ead-9650-406b20d7b364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:46.943618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514922b-4c3d-4309-aeb7-a75b729fa1ba None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:46.944235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514922b-4c3d-4309-aeb7-a75b729fa1ba None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:46.944727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514922b-4c3d-4309-aeb7-a75b729fa1ba None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:46.946258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 232/457] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:09:46.949305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 226/458] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:46.952346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4020a56-cfc6-405a-97cf-8c96028c3a64 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:46.952848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4020a56-cfc6-405a-97cf-8c96028c3a64 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:46.953345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4020a56-cfc6-405a-97cf-8c96028c3a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:46.962247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:46.973873 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a4020a56-cfc6-405a-97cf-8c96028c3a64 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:46.983645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a4020a56-cfc6-405a-97cf-8c96028c3a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:47.088724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 233/459] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:47.174443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b08a8a-3a6a-4f6b-a8a8-02c5fc2437ca None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:47.174724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b08a8a-3a6a-4f6b-a8a8-02c5fc2437ca None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:47.175002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b08a8a-3a6a-4f6b-a8a8-02c5fc2437ca None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:47.176612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 227/460] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:47.181944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fe5bc-8bdf-4f55-b774-b8a1c78059b9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:47.182223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fe5bc-8bdf-4f55-b774-b8a1c78059b9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:47.182492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fe5bc-8bdf-4f55-b774-b8a1c78059b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:47.208543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c64fe5bc-8bdf-4f55-b774-b8a1c78059b9 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:47.209726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c64fe5bc-8bdf-4f55-b774-b8a1c78059b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:47.268818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b08a8a-3a6a-4f6b-a8a8-02c5fc2437ca None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:47.290854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 234/461] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:47.433262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b08a8a-3a6a-4f6b-a8a8-02c5fc2437ca None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:47.456897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fe5bc-8bdf-4f55-b774-b8a1c78059b9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:47.481209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcde0a-e338-452d-9a9c-9b94e4231d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:47.482000 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcde0a-e338-452d-9a9c-9b94e4231d0d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:47.482805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcde0a-e338-452d-9a9c-9b94e4231d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:47.483072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcde0a-e338-452d-9a9c-9b94e4231d0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:47.489961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:47.505190 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcde0a-e338-452d-9a9c-9b94e4231d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:47.506644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcde0a-e338-452d-9a9c-9b94e4231d0d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-07fcde0a-e338-452d-9a9c-9b94e4231d0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:47.529527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:47.530494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:47.531362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:47.531362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:47.531362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:47.531362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 228/462] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:47.536765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcde0a-e338-452d-9a9c-9b94e4231d0d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:47.606567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:47.648550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcde0a-e338-452d-9a9c-9b94e4231d0d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:47.650076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fe5bc-8bdf-4f55-b774-b8a1c78059b9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:47.684844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3732ba3-b405-4e17-bb33-d76bc5c06b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:47.686022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3732ba3-b405-4e17-bb33-d76bc5c06b5d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:47.686300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3732ba3-b405-4e17-bb33-d76bc5c06b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:47.686613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3732ba3-b405-4e17-bb33-d76bc5c06b5d None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:47.687248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3732ba3-b405-4e17-bb33-d76bc5c06b5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:47.687573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3732ba3-b405-4e17-bb33-d76bc5c06b5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:47.687785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354af3d5-36a6-466b-8bf8-37e6ea861cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:47.688422 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af3d5-36a6-466b-8bf8-37e6ea861cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:47.688704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af3d5-36a6-466b-8bf8-37e6ea861cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:47.688958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3732ba3-b405-4e17-bb33-d76bc5c06b5d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:47.689114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af3d5-36a6-466b-8bf8-37e6ea861cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:47.707908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354af3d5-36a6-466b-8bf8-37e6ea861cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:47.709689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354af3d5-36a6-466b-8bf8-37e6ea861cf7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:47.715743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 229/463] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:47.716681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3732ba3-b405-4e17-bb33-d76bc5c06b5d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-354af3d5-36a6-466b-8bf8-37e6ea861cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:47.726464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:47.727470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:47.728334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:47.728334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:47.728334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:47.730580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 235/464] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:47.731583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354af3d5-36a6-466b-8bf8-37e6ea861cf7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:47.776816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:47.832965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3732ba3-b405-4e17-bb33-d76bc5c06b5d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:47.847478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354af3d5-36a6-466b-8bf8-37e6ea861cf7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:47.867073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:47.870116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9101078-e5f9-48ab-a790-b030d4fb72da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:47.870741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9101078-e5f9-48ab-a790-b030d4fb72da None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:47.871015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9101078-e5f9-48ab-a790-b030d4fb72da None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:47.871696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9101078-e5f9-48ab-a790-b030d4fb72da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:47.881764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9101078-e5f9-48ab-a790-b030d4fb72da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:47.883336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9101078-e5f9-48ab-a790-b030d4fb72da None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:47.886444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75c0ad-b6c9-4c12-901f-be7fc1803171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:47.887042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75c0ad-b6c9-4c12-901f-be7fc1803171 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:47.887318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75c0ad-b6c9-4c12-901f-be7fc1803171 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:47.887612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75c0ad-b6c9-4c12-901f-be7fc1803171 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:47.888249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75c0ad-b6c9-4c12-901f-be7fc1803171 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:47.889688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75c0ad-b6c9-4c12-901f-be7fc1803171 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-a9101078-e5f9-48ab-a790-b030d4fb72da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:47.898228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:47.899141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:47.900033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:47.900033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:47.900033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:47.900033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 230/465] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:47.900745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9101078-e5f9-48ab-a790-b030d4fb72da None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:47.959695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 236/466] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:09:47 2022] POST /identity/v3/users => generated 246 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:47.960918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f0f73-244e-4846-8605-0a63c3dc2293 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:47.961203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f0f73-244e-4846-8605-0a63c3dc2293 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:47.961470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f0f73-244e-4846-8605-0a63c3dc2293 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:47.962995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 237/467] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:47.963740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f0f73-244e-4846-8605-0a63c3dc2293 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:48.029495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:48.033997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9101078-e5f9-48ab-a790-b030d4fb72da None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:48.081248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce4319-9592-48c0-8306-d5fe33b1a948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:48.081866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce4319-9592-48c0-8306-d5fe33b1a948 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:48.082142 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce4319-9592-48c0-8306-d5fe33b1a948 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:48.082413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce4319-9592-48c0-8306-d5fe33b1a948 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:48.083055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce4319-9592-48c0-8306-d5fe33b1a948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:48.083352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce4319-9592-48c0-8306-d5fe33b1a948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:48.085285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce4319-9592-48c0-8306-d5fe33b1a948 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:48.101557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 231/468] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:48.105114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407caebb-813e-4cf7-a39e-6649bf3c582b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:48.105114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407caebb-813e-4cf7-a39e-6649bf3c582b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:48.105442 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407caebb-813e-4cf7-a39e-6649bf3c582b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:48.119512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f0f73-244e-4846-8605-0a63c3dc2293 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:48.132495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-407caebb-813e-4cf7-a39e-6649bf3c582b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:48.134043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-407caebb-813e-4cf7-a39e-6649bf3c582b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:48.166429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ab746-293a-4f86-8ca1-a027f6422e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:48.167411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ab746-293a-4f86-8ca1-a027f6422e95 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:48.167950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ab746-293a-4f86-8ca1-a027f6422e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:48.168390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ab746-293a-4f86-8ca1-a027f6422e95 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:48.169194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ab746-293a-4f86-8ca1-a027f6422e95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:48.169484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ab746-293a-4f86-8ca1-a027f6422e95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:48.171527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ab746-293a-4f86-8ca1-a027f6422e95 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:48.183785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 238/469] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:48.184520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ab746-293a-4f86-8ca1-a027f6422e95 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:48.219071 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 232/470] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:48.220119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407caebb-813e-4cf7-a39e-6649bf3c582b None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:48.227876 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:48.320135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ab746-293a-4f86-8ca1-a027f6422e95 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:48.331581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:48.343255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407caebb-813e-4cf7-a39e-6649bf3c582b None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:48.370201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97f562-9210-4070-a8ee-7af1258395ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:48.370827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97f562-9210-4070-a8ee-7af1258395ab None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:48.371110 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97f562-9210-4070-a8ee-7af1258395ab None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:48.374361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97f562-9210-4070-a8ee-7af1258395ab None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:48.375032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97f562-9210-4070-a8ee-7af1258395ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:48.376510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97f562-9210-4070-a8ee-7af1258395ab None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:48.410358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c927a-1f97-43c3-a6f5-76d1ba505d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:48.410971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c927a-1f97-43c3-a6f5-76d1ba505d99 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:48.411258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c927a-1f97-43c3-a6f5-76d1ba505d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:48.411672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c927a-1f97-43c3-a6f5-76d1ba505d99 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:48.412258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c927a-1f97-43c3-a6f5-76d1ba505d99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:48.413699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c927a-1f97-43c3-a6f5-76d1ba505d99 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:48.578508 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 239/471] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:09:48 2022] POST /identity/v3/users => generated 248 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:48.579902 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97f562-9210-4070-a8ee-7af1258395ab None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:48.607299 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 233/472] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:09:48 2022] POST /identity/v3/users => generated 248 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:48.683567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:48.703183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97f562-9210-4070-a8ee-7af1258395ab None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:48.709118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae5aa6-55cd-419b-b52c-e271cc2762ad None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:48.709462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae5aa6-55cd-419b-b52c-e271cc2762ad None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:48.709670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae5aa6-55cd-419b-b52c-e271cc2762ad None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:48.711102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 234/473] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:09:48.718072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3852-6c75-4e76-bfa4-9e49d11804dc None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:48.718404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3852-6c75-4e76-bfa4-9e49d11804dc None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:48.718629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3852-6c75-4e76-bfa4-9e49d11804dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:48.753135 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-128d3852-6c75-4e76-bfa4-9e49d11804dc None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:48.756521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-128d3852-6c75-4e76-bfa4-9e49d11804dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:48.770849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e126d1e-2893-4b28-8add-d8083ac46eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:48.771953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e126d1e-2893-4b28-8add-d8083ac46eee None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:48.772434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e126d1e-2893-4b28-8add-d8083ac46eee None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:48.772918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e126d1e-2893-4b28-8add-d8083ac46eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:48.795660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e126d1e-2893-4b28-8add-d8083ac46eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:48.797899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e126d1e-2893-4b28-8add-d8083ac46eee None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-8e126d1e-2893-4b28-8add-d8083ac46eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:48.818808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:48.819788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:48.821078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:48.821078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:48.821078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:48.821078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 240/474] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:48.828615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e126d1e-2893-4b28-8add-d8083ac46eee None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:48.876915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 235/475] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:48.935963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e126d1e-2893-4b28-8add-d8083ac46eee None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:48.975703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ab9c3-9ca0-4dc9-af1f-403cd6d526e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:48.976323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ab9c3-9ca0-4dc9-af1f-403cd6d526e2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:48.976610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ab9c3-9ca0-4dc9-af1f-403cd6d526e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:48.976888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ab9c3-9ca0-4dc9-af1f-403cd6d526e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:48.977605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ab9c3-9ca0-4dc9-af1f-403cd6d526e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:48.977909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ab9c3-9ca0-4dc9-af1f-403cd6d526e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:48.979339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ab9c3-9ca0-4dc9-af1f-403cd6d526e2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:48.990129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:48.994072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 241/476] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:49.000321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d3852-6c75-4e76-bfa4-9e49d11804dc None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:49.112807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:49.149247 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d3852-6c75-4e76-bfa4-9e49d11804dc None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:49.163189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ab9c3-9ca0-4dc9-af1f-403cd6d526e2 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:49.212639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30102ed5-8cd3-41c7-88c3-3746f895cde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:49.213249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30102ed5-8cd3-41c7-88c3-3746f895cde7 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:49.213527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30102ed5-8cd3-41c7-88c3-3746f895cde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:49.213873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30102ed5-8cd3-41c7-88c3-3746f895cde7 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:49.214444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30102ed5-8cd3-41c7-88c3-3746f895cde7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:49.219426 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30102ed5-8cd3-41c7-88c3-3746f895cde7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:49.298467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 236/477] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:09:48 2022] POST /identity/v3/users => generated 249 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:09:49.299366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ab9c3-9ca0-4dc9-af1f-403cd6d526e2 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:49.338313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f06103-9bfa-4aa4-88b8-54f2f4a4cda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:49.338941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f06103-9bfa-4aa4-88b8-54f2f4a4cda6 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:49.339218 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f06103-9bfa-4aa4-88b8-54f2f4a4cda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:49.339490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f06103-9bfa-4aa4-88b8-54f2f4a4cda6 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:49.340215 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f06103-9bfa-4aa4-88b8-54f2f4a4cda6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:49.342453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f06103-9bfa-4aa4-88b8-54f2f4a4cda6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:49.389403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.sql.core [None req-f4f06103-9bfa-4aa4-88b8-54f2f4a4cda6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 12:09:49.389403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 05 12:09:49.389403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [parameters: {'id': '844f6ee8c12448a7a9b62fbb1c086753', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 05 12:09:49.389403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81649) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f4f06103-9bfa-4aa4-88b8-54f2f4a4cda6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result.read() Sep 05 12:09:49.414337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return ref.to_dict() Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:09:49.415278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application yield resource Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise self.value Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application session.commit() Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 12:09:49.416289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.session.flush() Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application compat.raise_( Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 12:09:49.417155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:09:49.418074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application result.read() Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:09:49.418914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [parameters: {'id': '844f6ee8c12448a7a9b62fbb1c086753', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:49.421913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 05 12:09:49.430019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:49.432109 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 242/478] 10.222.0.62 () {60 vars in 1131 bytes} [Mon Sep 5 12:09:49 2022] POST /identity/v3/roles => generated 144 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:49.432264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30102ed5-8cd3-41c7-88c3-3746f895cde7 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:49.561180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30102ed5-8cd3-41c7-88c3-3746f895cde7 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:49.590823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:49.597845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97082850-d131-435f-9b5c-af98f338dcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:49.598473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97082850-d131-435f-9b5c-af98f338dcfd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:49.598747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97082850-d131-435f-9b5c-af98f338dcfd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:49.599017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97082850-d131-435f-9b5c-af98f338dcfd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:49.610292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97082850-d131-435f-9b5c-af98f338dcfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-97082850-d131-435f-9b5c-af98f338dcfd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:49.615030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:49.618812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:49.620838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 237/479] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:09:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:49.624121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f06103-9bfa-4aa4-88b8-54f2f4a4cda6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:49.706694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:49.745787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f06103-9bfa-4aa4-88b8-54f2f4a4cda6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:49.782828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf9477-c580-414a-b454-58f80fad937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:49.783445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf9477-c580-414a-b454-58f80fad937c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:49.783747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf9477-c580-414a-b454-58f80fad937c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:49.784019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf9477-c580-414a-b454-58f80fad937c None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:49.784690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf9477-c580-414a-b454-58f80fad937c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:49.784987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf9477-c580-414a-b454-58f80fad937c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:49.786343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf9477-c580-414a-b454-58f80fad937c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:49.794786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 243/480] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:50.360635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038da5ac-56bc-48bd-9dde-97e2564cbeba None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:50.361129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038da5ac-56bc-48bd-9dde-97e2564cbeba None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:50.361129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038da5ac-56bc-48bd-9dde-97e2564cbeba None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:50.362688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 238/481] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:50.368058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96804e9-a906-4fce-a1af-232e9076a6fa None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:50.368448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96804e9-a906-4fce-a1af-232e9076a6fa None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:50.368611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96804e9-a906-4fce-a1af-232e9076a6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:50.395329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f96804e9-a906-4fce-a1af-232e9076a6fa None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:50.396564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f96804e9-a906-4fce-a1af-232e9076a6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:50.495814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 244/482] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:09:51.018564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1418c9f-2cf5-4687-ae00-d68fe3c2037c None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:51.018949 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1418c9f-2cf5-4687-ae00-d68fe3c2037c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:51.019135 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1418c9f-2cf5-4687-ae00-d68fe3c2037c None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:51.020946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 239/483] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:51.026240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90340bf-4e88-4e71-beec-025f9bdd58ae None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:51.026529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90340bf-4e88-4e71-beec-025f9bdd58ae None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:51.026799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90340bf-4e88-4e71-beec-025f9bdd58ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:51.043038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1418c9f-2cf5-4687-ae00-d68fe3c2037c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:51.047251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a90340bf-4e88-4e71-beec-025f9bdd58ae None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:51.050673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a90340bf-4e88-4e71-beec-025f9bdd58ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:51.145927 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:51.171663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 245/484] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:51.173250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e571cb-e88f-4d74-a52a-2e1d06c372c1 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:51.173726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e571cb-e88f-4d74-a52a-2e1d06c372c1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:51.174195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e571cb-e88f-4d74-a52a-2e1d06c372c1 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:51.176762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 246/485] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:51.191559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf11f8-7a6f-4692-ade6-db2ef5b2659e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:51.191559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf11f8-7a6f-4692-ade6-db2ef5b2659e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:51.191559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf11f8-7a6f-4692-ade6-db2ef5b2659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:51.209084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1418c9f-2cf5-4687-ae00-d68fe3c2037c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:51.214792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-47bf11f8-7a6f-4692-ade6-db2ef5b2659e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:51.214792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-47bf11f8-7a6f-4692-ade6-db2ef5b2659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:51.271082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142aeca5-6fb9-42d9-a334-20d0081f4bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:51.272039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142aeca5-6fb9-42d9-a334-20d0081f4bbd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:51.272546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142aeca5-6fb9-42d9-a334-20d0081f4bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:51.273007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142aeca5-6fb9-42d9-a334-20d0081f4bbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:51.283990 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142aeca5-6fb9-42d9-a334-20d0081f4bbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-142aeca5-6fb9-42d9-a334-20d0081f4bbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:51.289889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:51.290760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:51.291602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:51.294193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 240/486] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:51.300564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142aeca5-6fb9-42d9-a334-20d0081f4bbd None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:51.357763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 247/487] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:09:51.371795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:51.435083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142aeca5-6fb9-42d9-a334-20d0081f4bbd None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:51.475087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4954a-4f44-4b03-8a88-2f923e4f96fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:51.475732 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4954a-4f44-4b03-8a88-2f923e4f96fc None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:51.476012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4954a-4f44-4b03-8a88-2f923e4f96fc None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:51.476281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4954a-4f44-4b03-8a88-2f923e4f96fc None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:51.476940 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4954a-4f44-4b03-8a88-2f923e4f96fc None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:51.477238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4954a-4f44-4b03-8a88-2f923e4f96fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:51.478607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4954a-4f44-4b03-8a88-2f923e4f96fc None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:51.491998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 241/488] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:51.493270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf11f8-7a6f-4692-ade6-db2ef5b2659e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:51.541803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4954a-4f44-4b03-8a88-2f923e4f96fc None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:51.647745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4954a-4f44-4b03-8a88-2f923e4f96fc None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:51.662965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:51.683957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a76cc-f25a-414c-8c57-645ae994a180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:51.684979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a76cc-f25a-414c-8c57-645ae994a180 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:51.685578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a76cc-f25a-414c-8c57-645ae994a180 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:51.686041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a76cc-f25a-414c-8c57-645ae994a180 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:51.692060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf11f8-7a6f-4692-ade6-db2ef5b2659e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:51.697125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a76cc-f25a-414c-8c57-645ae994a180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-927a76cc-f25a-414c-8c57-645ae994a180 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:51.702831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:51.703753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:51.704554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:51.706614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 242/489] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:51.707212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a76cc-f25a-414c-8c57-645ae994a180 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:51.719226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:51.738740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a7686-fc3c-48d0-889f-7a5278317934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:51.739619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a7686-fc3c-48d0-889f-7a5278317934 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:51.739908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a7686-fc3c-48d0-889f-7a5278317934 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:51.740096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a7686-fc3c-48d0-889f-7a5278317934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:51.751376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a7686-fc3c-48d0-889f-7a5278317934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:51.752897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a7686-fc3c-48d0-889f-7a5278317934 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-b12a7686-fc3c-48d0-889f-7a5278317934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:51.769086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:51.770038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:51.771489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:51.771489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:51.771489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:51.771489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 248/490] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:51.771489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a7686-fc3c-48d0-889f-7a5278317934 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:51.824012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a76cc-f25a-414c-8c57-645ae994a180 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:51.862051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b6d2-d950-413c-aae1-028107ed28af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:51.863133 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b6d2-d950-413c-aae1-028107ed28af None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:51.863635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b6d2-d950-413c-aae1-028107ed28af None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:51.864120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b6d2-d950-413c-aae1-028107ed28af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:51.875206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4b6d2-d950-413c-aae1-028107ed28af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-eed4b6d2-d950-413c-aae1-028107ed28af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:51.880943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:51.881845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:51.882666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:51.883276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a7686-fc3c-48d0-889f-7a5278317934 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:51.885075 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 243/491] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:51.886176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b6d2-d950-413c-aae1-028107ed28af None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:51.923788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:51.925986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e43a0f-5727-48f5-836f-5ef0a212d006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:51.925986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e43a0f-5727-48f5-836f-5ef0a212d006 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:51.926200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e43a0f-5727-48f5-836f-5ef0a212d006 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:51.926392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e43a0f-5727-48f5-836f-5ef0a212d006 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:51.927042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e43a0f-5727-48f5-836f-5ef0a212d006 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:51.927343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e43a0f-5727-48f5-836f-5ef0a212d006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:51.928818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e43a0f-5727-48f5-836f-5ef0a212d006 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:51.937402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 249/492] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:51.938221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e43a0f-5727-48f5-836f-5ef0a212d006 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:51.971903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:51.999983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b6d2-d950-413c-aae1-028107ed28af None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:52.035968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e9e72-26b7-438c-9a5c-077b32c8b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:52.036566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e9e72-26b7-438c-9a5c-077b32c8b930 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.036847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e9e72-26b7-438c-9a5c-077b32c8b930 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.037117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e9e72-26b7-438c-9a5c-077b32c8b930 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.037742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e9e72-26b7-438c-9a5c-077b32c8b930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:52.038039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e9e72-26b7-438c-9a5c-077b32c8b930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:52.039420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e9e72-26b7-438c-9a5c-077b32c8b930 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:52.055388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 244/493] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:52.055802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e43a0f-5727-48f5-836f-5ef0a212d006 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:52.056147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e9e72-26b7-438c-9a5c-077b32c8b930 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:52.093868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16378d40-f783-492a-abaa-caa4bd48d4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:52.094402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378d40-f783-492a-abaa-caa4bd48d4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.094677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378d40-f783-492a-abaa-caa4bd48d4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.094962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378d40-f783-492a-abaa-caa4bd48d4d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.098021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16378d40-f783-492a-abaa-caa4bd48d4d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:52.098021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16378d40-f783-492a-abaa-caa4bd48d4d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:52.099063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16378d40-f783-492a-abaa-caa4bd48d4d4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:52.107857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 250/494] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:52.109650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d899-4571-4a34-8baa-04a43448eeff None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.110141 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d899-4571-4a34-8baa-04a43448eeff None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.110821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d899-4571-4a34-8baa-04a43448eeff None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.112880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 251/495] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:52.113583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97d899-4571-4a34-8baa-04a43448eeff None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:52.170501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:52.177871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e9e72-26b7-438c-9a5c-077b32c8b930 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:52.208629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:52.213227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b23fa-579d-45b9-b2cc-c13aba803a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:52.213859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b23fa-579d-45b9-b2cc-c13aba803a0f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.214140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b23fa-579d-45b9-b2cc-c13aba803a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.214422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b23fa-579d-45b9-b2cc-c13aba803a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.218953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97d899-4571-4a34-8baa-04a43448eeff None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:52.225483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b23fa-579d-45b9-b2cc-c13aba803a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:52.226975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b23fa-579d-45b9-b2cc-c13aba803a0f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-385b23fa-579d-45b9-b2cc-c13aba803a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:52.242432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:52.243340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:52.244212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:52.244212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:52.244212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:52.244212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 245/496] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:52.244446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b23fa-579d-45b9-b2cc-c13aba803a0f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:52.260920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a489810-48b4-4cb3-b489-3c5c653086a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:52.261930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a489810-48b4-4cb3-b489-3c5c653086a8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.262361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a489810-48b4-4cb3-b489-3c5c653086a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.262643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a489810-48b4-4cb3-b489-3c5c653086a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.270405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a489810-48b4-4cb3-b489-3c5c653086a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:52.271869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a489810-48b4-4cb3-b489-3c5c653086a8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-4a489810-48b4-4cb3-b489-3c5c653086a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:52.285107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:52.286015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 252/497] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Sep 5 12:09:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1025c8b-146e-4d3b-89e0-5b3be63e527a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.291859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1025c8b-146e-4d3b-89e0-5b3be63e527a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.292559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1025c8b-146e-4d3b-89e0-5b3be63e527a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.307099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b1025c8b-146e-4d3b-89e0-5b3be63e527a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:52.308329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b1025c8b-146e-4d3b-89e0-5b3be63e527a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:52.358332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b23fa-579d-45b9-b2cc-c13aba803a0f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:52.379284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 253/498] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:52.379959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1025c8b-146e-4d3b-89e0-5b3be63e527a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:52.403849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf05a0-eef5-4d2b-93f8-6ade49382f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:52.405964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf05a0-eef5-4d2b-93f8-6ade49382f0a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.406711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf05a0-eef5-4d2b-93f8-6ade49382f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.407203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf05a0-eef5-4d2b-93f8-6ade49382f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.422301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf05a0-eef5-4d2b-93f8-6ade49382f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:52.424508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf05a0-eef5-4d2b-93f8-6ade49382f0a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:52.434718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-72cf05a0-eef5-4d2b-93f8-6ade49382f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:52.443372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:52.444313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:52.445821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:52.445821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:52.445821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:52.445821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 246/499] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:52.445821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf05a0-eef5-4d2b-93f8-6ade49382f0a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:52.478463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:52.499008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1025c8b-146e-4d3b-89e0-5b3be63e527a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:52.534977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ec51d-7918-4f7d-a78c-7b5ecd0170ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:52.535607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec51d-7918-4f7d-a78c-7b5ecd0170ae None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.535879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec51d-7918-4f7d-a78c-7b5ecd0170ae None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.536147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec51d-7918-4f7d-a78c-7b5ecd0170ae None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.536780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ec51d-7918-4f7d-a78c-7b5ecd0170ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:52.537074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ec51d-7918-4f7d-a78c-7b5ecd0170ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:52.538500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ec51d-7918-4f7d-a78c-7b5ecd0170ae None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:52.551661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 254/500] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:52.551661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ec51d-7918-4f7d-a78c-7b5ecd0170ae None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:52.551874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf05a0-eef5-4d2b-93f8-6ade49382f0a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:52.591774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa79adf-bea5-43e7-99de-9b55f82f9203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:52.592385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa79adf-bea5-43e7-99de-9b55f82f9203 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.592666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa79adf-bea5-43e7-99de-9b55f82f9203 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.592941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa79adf-bea5-43e7-99de-9b55f82f9203 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.593669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa79adf-bea5-43e7-99de-9b55f82f9203 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:52.593975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa79adf-bea5-43e7-99de-9b55f82f9203 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:52.595414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa79adf-bea5-43e7-99de-9b55f82f9203 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:52.632139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 247/501] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Sep 5 12:09:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:52.633342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad271e0-e334-41c0-a9be-c1caba87e7c8 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.633631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad271e0-e334-41c0-a9be-c1caba87e7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.633906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad271e0-e334-41c0-a9be-c1caba87e7c8 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.635114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 248/502] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:52.639569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad271e0-e334-41c0-a9be-c1caba87e7c8 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:52.662157 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ec51d-7918-4f7d-a78c-7b5ecd0170ae None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:52.695104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8664b42-b52e-44e0-9319-a992a8b4ce32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:52.695738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8664b42-b52e-44e0-9319-a992a8b4ce32 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.696017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8664b42-b52e-44e0-9319-a992a8b4ce32 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.696299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8664b42-b52e-44e0-9319-a992a8b4ce32 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.696447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:52.696938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8664b42-b52e-44e0-9319-a992a8b4ce32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:52.697230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8664b42-b52e-44e0-9319-a992a8b4ce32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:52.698608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8664b42-b52e-44e0-9319-a992a8b4ce32 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:52.710697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 255/503] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:52.711489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8664b42-b52e-44e0-9319-a992a8b4ce32 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:52.714283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:52.742642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad271e0-e334-41c0-a9be-c1caba87e7c8 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:52.775927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84211a78-8789-4907-8527-4625e553c1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:52.776542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84211a78-8789-4907-8527-4625e553c1bd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.776825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84211a78-8789-4907-8527-4625e553c1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.777094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84211a78-8789-4907-8527-4625e553c1bd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.784278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84211a78-8789-4907-8527-4625e553c1bd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:52.785731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84211a78-8789-4907-8527-4625e553c1bd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-84211a78-8789-4907-8527-4625e553c1bd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:52.798612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:52.799882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:52.800983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:52.800983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:52.800983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:52.800983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:52.800983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:52.800983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:52.800983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:52.800983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:52.800983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:09:52.800983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:52.803822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 249/504] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:52.805070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a52fd-8006-4b91-aca2-f0b78c462cf0 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.805355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a52fd-8006-4b91-aca2-f0b78c462cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.805628 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a52fd-8006-4b91-aca2-f0b78c462cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.811504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8664b42-b52e-44e0-9319-a992a8b4ce32 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:52.823467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-784a52fd-8006-4b91-aca2-f0b78c462cf0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:52.824658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-784a52fd-8006-4b91-aca2-f0b78c462cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:52.844531 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201d1e7-5781-4b00-a265-b080e30b5db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:52.845130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d1e7-5781-4b00-a265-b080e30b5db5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:52.845405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d1e7-5781-4b00-a265-b080e30b5db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:52.845672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d1e7-5781-4b00-a265-b080e30b5db5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:52.890122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 250/505] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:52.891118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a52fd-8006-4b91-aca2-f0b78c462cf0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:52.911856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201d1e7-5781-4b00-a265-b080e30b5db5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:52.913369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201d1e7-5781-4b00-a265-b080e30b5db5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:52.915234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:52.941769 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:53.006032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a52fd-8006-4b91-aca2-f0b78c462cf0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.029573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 256/506] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:53.029573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201d1e7-5781-4b00-a265-b080e30b5db5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.039935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96957-ac16-4229-8f3d-614a98673cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.040569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96957-ac16-4229-8f3d-614a98673cbe None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.040850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96957-ac16-4229-8f3d-614a98673cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.041116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96957-ac16-4229-8f3d-614a98673cbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.054880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f96957-ac16-4229-8f3d-614a98673cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-11f96957-ac16-4229-8f3d-614a98673cbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:53.059755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:53.061452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:53.062443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:53.063614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 251/507] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:53.068620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96957-ac16-4229-8f3d-614a98673cbe None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.144927 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201d1e7-5781-4b00-a265-b080e30b5db5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.146579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:53.177711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96957-ac16-4229-8f3d-614a98673cbe None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.179958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899a960-9984-4d63-b981-f742cdd12527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.180672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899a960-9984-4d63-b981-f742cdd12527 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.180954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899a960-9984-4d63-b981-f742cdd12527 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.181226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899a960-9984-4d63-b981-f742cdd12527 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.184999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:53.189192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899a960-9984-4d63-b981-f742cdd12527 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:53.190906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899a960-9984-4d63-b981-f742cdd12527 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-5899a960-9984-4d63-b981-f742cdd12527 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:53.202940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:53.204051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:53.204931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 257/508] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:53.205523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899a960-9984-4d63-b981-f742cdd12527 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.215891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7741-abdc-4ba9-ae6a-49b4bd9f4e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.216496 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7741-abdc-4ba9-ae6a-49b4bd9f4e08 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.216807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7741-abdc-4ba9-ae6a-49b4bd9f4e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.217074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7741-abdc-4ba9-ae6a-49b4bd9f4e08 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.217792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7741-abdc-4ba9-ae6a-49b4bd9f4e08 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:53.218092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7741-abdc-4ba9-ae6a-49b4bd9f4e08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:53.219561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7741-abdc-4ba9-ae6a-49b4bd9f4e08 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:53.236866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 252/509] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:53.237638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7741-abdc-4ba9-ae6a-49b4bd9f4e08 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.303134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899a960-9984-4d63-b981-f742cdd12527 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.336109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7741-abdc-4ba9-ae6a-49b4bd9f4e08 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.336884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897d7b4-53a7-4eab-af20-7a844b736ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.337764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d7b4-53a7-4eab-af20-7a844b736ccd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.338038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d7b4-53a7-4eab-af20-7a844b736ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.338309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d7b4-53a7-4eab-af20-7a844b736ccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.348990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897d7b4-53a7-4eab-af20-7a844b736ccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:53.350340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897d7b4-53a7-4eab-af20-7a844b736ccd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-8897d7b4-53a7-4eab-af20-7a844b736ccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:53.364316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:53.365418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:53.366467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:53.366467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:53.366467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:53.366467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 258/510] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:53.366467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897d7b4-53a7-4eab-af20-7a844b736ccd None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.371691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09425ecd-3eed-4d52-93ae-e55996f07a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.372297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09425ecd-3eed-4d52-93ae-e55996f07a55 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.372575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09425ecd-3eed-4d52-93ae-e55996f07a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.372846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09425ecd-3eed-4d52-93ae-e55996f07a55 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.374012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09425ecd-3eed-4d52-93ae-e55996f07a55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:53.374314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09425ecd-3eed-4d52-93ae-e55996f07a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:53.375717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09425ecd-3eed-4d52-93ae-e55996f07a55 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:53.382383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:53.383738 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 253/511] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:53.384336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09425ecd-3eed-4d52-93ae-e55996f07a55 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.411688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:53.470803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897d7b4-53a7-4eab-af20-7a844b736ccd None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.483801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09425ecd-3eed-4d52-93ae-e55996f07a55 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.505001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ea9c5-7c9d-416f-b306-bf64c90bb30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.505625 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ea9c5-7c9d-416f-b306-bf64c90bb30d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.505901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ea9c5-7c9d-416f-b306-bf64c90bb30d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.506170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ea9c5-7c9d-416f-b306-bf64c90bb30d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.516970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ea9c5-7c9d-416f-b306-bf64c90bb30d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:53.517336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75589c5-d787-4fd5-b78f-e0ca71992cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.517925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75589c5-d787-4fd5-b78f-e0ca71992cac None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.518200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75589c5-d787-4fd5-b78f-e0ca71992cac None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.518394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ea9c5-7c9d-416f-b306-bf64c90bb30d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:53.518493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75589c5-d787-4fd5-b78f-e0ca71992cac None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.519164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75589c5-d787-4fd5-b78f-e0ca71992cac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:53.519461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75589c5-d787-4fd5-b78f-e0ca71992cac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:53.520912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75589c5-d787-4fd5-b78f-e0ca71992cac None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-580ea9c5-7c9d-416f-b306-bf64c90bb30d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:53.532740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:53.533810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:53.534698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:53.534698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:53.534698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:53.534698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 259/512] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:53.534698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ea9c5-7c9d-416f-b306-bf64c90bb30d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.538726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 254/513] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:53.539520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75589c5-d787-4fd5-b78f-e0ca71992cac None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.601762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:53.625296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:53.635920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ea9c5-7c9d-416f-b306-bf64c90bb30d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.641261 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75589c5-d787-4fd5-b78f-e0ca71992cac None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.669831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e917d80-3571-4a8b-a8f7-7381abf23fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.670391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e917d80-3571-4a8b-a8f7-7381abf23fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.670668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e917d80-3571-4a8b-a8f7-7381abf23fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.670937 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e917d80-3571-4a8b-a8f7-7381abf23fe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.674687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d610196-c27a-4866-b428-1f68f8614c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.675278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d610196-c27a-4866-b428-1f68f8614c4c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.675584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d610196-c27a-4866-b428-1f68f8614c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.675849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d610196-c27a-4866-b428-1f68f8614c4c None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.676474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d610196-c27a-4866-b428-1f68f8614c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:53.676768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d610196-c27a-4866-b428-1f68f8614c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:53.678141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d610196-c27a-4866-b428-1f68f8614c4c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:53.688924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 255/514] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:53.689683 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d610196-c27a-4866-b428-1f68f8614c4c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.737630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e917d80-3571-4a8b-a8f7-7381abf23fe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:53.739072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e917d80-3571-4a8b-a8f7-7381abf23fe5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:53.786349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d610196-c27a-4866-b428-1f68f8614c4c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.801215 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:53.819607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e582-67b3-4383-b584-c537b32abe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.820222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e582-67b3-4383-b584-c537b32abe59 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.820498 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e582-67b3-4383-b584-c537b32abe59 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.820769 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e582-67b3-4383-b584-c537b32abe59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.832082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291e582-67b3-4383-b584-c537b32abe59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:53.833512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291e582-67b3-4383-b584-c537b32abe59 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:53.835556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 260/515] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:53.836012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:53.836573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e917d80-3571-4a8b-a8f7-7381abf23fe5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-3291e582-67b3-4383-b584-c537b32abe59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:53.852720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:53.853771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:53.854643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:53.854643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:53.854643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:53.854643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 256/516] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:53.854643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e582-67b3-4383-b584-c537b32abe59 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.936761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e917d80-3571-4a8b-a8f7-7381abf23fe5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.956344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e582-67b3-4383-b584-c537b32abe59 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:53.970164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1094e8-0799-43ac-9b34-e6890b5ad4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.970763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1094e8-0799-43ac-9b34-e6890b5ad4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.971039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1094e8-0799-43ac-9b34-e6890b5ad4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.971308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1094e8-0799-43ac-9b34-e6890b5ad4b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:53.971980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1094e8-0799-43ac-9b34-e6890b5ad4b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:53.972278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1094e8-0799-43ac-9b34-e6890b5ad4b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:53.973639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1094e8-0799-43ac-9b34-e6890b5ad4b7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:53.984717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 261/517] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:53.985498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1094e8-0799-43ac-9b34-e6890b5ad4b7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:53.994684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42948420-48b2-487d-ac82-1295413a49e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:53.995296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42948420-48b2-487d-ac82-1295413a49e4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:53.995602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42948420-48b2-487d-ac82-1295413a49e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:53.995881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42948420-48b2-487d-ac82-1295413a49e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.012720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:54.058866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:54.065907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42948420-48b2-487d-ac82-1295413a49e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.067404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42948420-48b2-487d-ac82-1295413a49e4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:54.088665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1094e8-0799-43ac-9b34-e6890b5ad4b7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:54.120987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc2dfc-854b-4678-95ad-4b18dd65f731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:54.121613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc2dfc-854b-4678-95ad-4b18dd65f731 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:54.121890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc2dfc-854b-4678-95ad-4b18dd65f731 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:54.122160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc2dfc-854b-4678-95ad-4b18dd65f731 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.129227 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc2dfc-854b-4678-95ad-4b18dd65f731 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.130667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc2dfc-854b-4678-95ad-4b18dd65f731 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-2dcc2dfc-854b-4678-95ad-4b18dd65f731 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:09:54.142869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:54.143786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:54.144657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:09:54.144657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:09:54.144657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:09:54.144657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:09:54.144657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:09:54.144657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:09:54.144657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:54.144657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:54.144657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 262/518] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:54.145131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc2dfc-854b-4678-95ad-4b18dd65f731 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:54.169678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 257/519] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:54.170534 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42948420-48b2-487d-ac82-1295413a49e4 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:54.228396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:54.243228 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc2dfc-854b-4678-95ad-4b18dd65f731 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:54.266825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:54.272787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42948420-48b2-487d-ac82-1295413a49e4 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:54.277563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c4e18-cf28-4ed8-b926-5b781aa2d4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:54.278186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c4e18-cf28-4ed8-b926-5b781aa2d4cf None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:54.278464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c4e18-cf28-4ed8-b926-5b781aa2d4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:54.278735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c4e18-cf28-4ed8-b926-5b781aa2d4cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.289380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c4e18-cf28-4ed8-b926-5b781aa2d4cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.290811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c4e18-cf28-4ed8-b926-5b781aa2d4cf None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-147c4e18-cf28-4ed8-b926-5b781aa2d4cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:54.305756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:54.306636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:54.307496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:54.307496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:54.307496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:54.307685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1564536-ca30-4393-aa92-b55e046182dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:54.307685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1564536-ca30-4393-aa92-b55e046182dd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:54.307848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 263/520] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:54.307914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1564536-ca30-4393-aa92-b55e046182dd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:54.307914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1564536-ca30-4393-aa92-b55e046182dd None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.308029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c4e18-cf28-4ed8-b926-5b781aa2d4cf None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:54.308094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1564536-ca30-4393-aa92-b55e046182dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:54.308293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1564536-ca30-4393-aa92-b55e046182dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.309578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1564536-ca30-4393-aa92-b55e046182dd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:54.321864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 258/521] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:54.322643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1564536-ca30-4393-aa92-b55e046182dd None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:54.411865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c4e18-cf28-4ed8-b926-5b781aa2d4cf None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:54.421821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1564536-ca30-4393-aa92-b55e046182dd None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:54.446322 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3539a-1f04-4c27-9cbe-8d5aa648a6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:54.446925 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3539a-1f04-4c27-9cbe-8d5aa648a6ed None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:54.447209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3539a-1f04-4c27-9cbe-8d5aa648a6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:54.447484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3539a-1f04-4c27-9cbe-8d5aa648a6ed None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.448152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3539a-1f04-4c27-9cbe-8d5aa648a6ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.449568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3539a-1f04-4c27-9cbe-8d5aa648a6ed None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:54.455140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01066ab-5316-4636-b232-cf1bd35e15b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:54.455768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01066ab-5316-4636-b232-cf1bd35e15b8 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:54.456048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01066ab-5316-4636-b232-cf1bd35e15b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:54.456318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01066ab-5316-4636-b232-cf1bd35e15b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.457003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01066ab-5316-4636-b232-cf1bd35e15b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:54.457302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01066ab-5316-4636-b232-cf1bd35e15b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.458756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01066ab-5316-4636-b232-cf1bd35e15b8 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:54.463218 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:54.476720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 259/522] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:09:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:54.477528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01066ab-5316-4636-b232-cf1bd35e15b8 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:54.495196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:54.573339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01066ab-5316-4636-b232-cf1bd35e15b8 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:54.582764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 264/523] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:09:54 2022] POST /identity/v3/users => generated 247 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:09:54.583684 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3539a-1f04-4c27-9cbe-8d5aa648a6ed None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:54.611676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c23065-0951-43d9-9e34-bce632fb19f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:54.612376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23065-0951-43d9-9e34-bce632fb19f1 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:54.612656 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23065-0951-43d9-9e34-bce632fb19f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:54.612925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23065-0951-43d9-9e34-bce632fb19f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.624786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c23065-0951-43d9-9e34-bce632fb19f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.626380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c23065-0951-43d9-9e34-bce632fb19f1 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-99c23065-0951-43d9-9e34-bce632fb19f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:54.643144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:54.644480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:54.645333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:54.645333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:54.645333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:54.649348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 260/524] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:54.650180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c23065-0951-43d9-9e34-bce632fb19f1 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:54.707967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3539a-1f04-4c27-9cbe-8d5aa648a6ed None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:54.724809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:54.748498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523b639-aecf-4b8a-a187-066c10b7b33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:54.749313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523b639-aecf-4b8a-a187-066c10b7b33b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:54.749738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523b639-aecf-4b8a-a187-066c10b7b33b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:54.750181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523b639-aecf-4b8a-a187-066c10b7b33b None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.751573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523b639-aecf-4b8a-a187-066c10b7b33b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:54.751573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523b639-aecf-4b8a-a187-066c10b7b33b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.752577 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523b639-aecf-4b8a-a187-066c10b7b33b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:54.763906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c23065-0951-43d9-9e34-bce632fb19f1 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:54.767455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 265/525] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:54.767766 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523b639-aecf-4b8a-a187-066c10b7b33b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:54.802652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796329e-adaa-404e-8b51-f39eec7db3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:54.803283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796329e-adaa-404e-8b51-f39eec7db3ee None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:54.803586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796329e-adaa-404e-8b51-f39eec7db3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:54.803868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796329e-adaa-404e-8b51-f39eec7db3ee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.811475 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796329e-adaa-404e-8b51-f39eec7db3ee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.812968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796329e-adaa-404e-8b51-f39eec7db3ee None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-4796329e-adaa-404e-8b51-f39eec7db3ee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:54.824271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:54.825251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:54.826196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 261/526] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:54.826776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796329e-adaa-404e-8b51-f39eec7db3ee None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:54.830410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:54.881455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523b639-aecf-4b8a-a187-066c10b7b33b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:54.919293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be9426-1d57-4391-a975-f2c887b75b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:54.919988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be9426-1d57-4391-a975-f2c887b75b2f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:54.920752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be9426-1d57-4391-a975-f2c887b75b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:54.921020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be9426-1d57-4391-a975-f2c887b75b2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.921864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be9426-1d57-4391-a975-f2c887b75b2f None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=7956a7748ec94a618eb466f5536220b0) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:54.922166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be9426-1d57-4391-a975-f2c887b75b2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.923604 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be9426-1d57-4391-a975-f2c887b75b2f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:54.930760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 266/527] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:54 2022] GET /identity/v3/role_assignments?user.id=7956a7748ec94a618eb466f5536220b0&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:54.931597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be9426-1d57-4391-a975-f2c887b75b2f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:54.934876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796329e-adaa-404e-8b51-f39eec7db3ee None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:54.976863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:54.986009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d41025-9895-46ce-9133-a0083d4f3a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:54.986765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d41025-9895-46ce-9133-a0083d4f3a6d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:54.991335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d41025-9895-46ce-9133-a0083d4f3a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:54.991335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d41025-9895-46ce-9133-a0083d4f3a6d None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:54.991335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d41025-9895-46ce-9133-a0083d4f3a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:54.991335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d41025-9895-46ce-9133-a0083d4f3a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:54.991335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d41025-9895-46ce-9133-a0083d4f3a6d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:55.003691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 262/528] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:55.004666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d41025-9895-46ce-9133-a0083d4f3a6d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:55.091517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be9426-1d57-4391-a975-f2c887b75b2f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:55.111662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:55.129138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea10d2-795b-453a-99f4-53f76f03409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:55.129807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea10d2-795b-453a-99f4-53f76f03409d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:55.130083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea10d2-795b-453a-99f4-53f76f03409d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:55.130351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea10d2-795b-453a-99f4-53f76f03409d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:55.139630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea10d2-795b-453a-99f4-53f76f03409d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:55.141656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea10d2-795b-453a-99f4-53f76f03409d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:55.146575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d41025-9895-46ce-9133-a0083d4f3a6d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-0eea10d2-795b-453a-99f4-53f76f03409d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:09:55.156970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:55.157848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:55.158695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:09:55.158695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:09:55.158695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:09:55.158695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:09:55.158695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:09:55.158695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:09:55.158695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:55.158695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:55.158695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 267/529] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:55.159161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea10d2-795b-453a-99f4-53f76f03409d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:55.195685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e10d3-fd50-4421-862b-876367050d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:55.196960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e10d3-fd50-4421-862b-876367050d00 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:55.197687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e10d3-fd50-4421-862b-876367050d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:55.198209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e10d3-fd50-4421-862b-876367050d00 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:55.199863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e10d3-fd50-4421-862b-876367050d00 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:55.200384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e10d3-fd50-4421-862b-876367050d00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:55.203666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e10d3-fd50-4421-862b-876367050d00 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:55.231521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 263/530] 10.222.0.62 () {56 vars in 1139 bytes} [Mon Sep 5 12:09:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:55.232775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e10d3-fd50-4421-862b-876367050d00 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:55.288978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea10d2-795b-453a-99f4-53f76f03409d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:55.299036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:55.333569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc7378-777d-4ccb-ba0a-3ccdda042dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:55.334233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc7378-777d-4ccb-ba0a-3ccdda042dfa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:55.334511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc7378-777d-4ccb-ba0a-3ccdda042dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:55.334789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc7378-777d-4ccb-ba0a-3ccdda042dfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:55.344339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc7378-777d-4ccb-ba0a-3ccdda042dfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:55.346485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc7378-777d-4ccb-ba0a-3ccdda042dfa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-28fc7378-777d-4ccb-ba0a-3ccdda042dfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:09:55.361176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:55.362036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:55.362888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:09:55.362888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:09:55.362888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:09:55.362888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:09:55.362888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:09:55.362888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:09:55.362888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:55.362888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:55.362888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 268/531] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:55.363836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc7378-777d-4ccb-ba0a-3ccdda042dfa None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:55.369111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e10d3-fd50-4421-862b-876367050d00 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:55.411135 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7ed67-15d8-4125-99f1-bb3a18655fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:55.411769 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7ed67-15d8-4125-99f1-bb3a18655fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:55.412045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7ed67-15d8-4125-99f1-bb3a18655fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:55.412312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7ed67-15d8-4125-99f1-bb3a18655fe9 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:55.412996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7ed67-15d8-4125-99f1-bb3a18655fe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:55.413304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7ed67-15d8-4125-99f1-bb3a18655fe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:55.414121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:55.415820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7ed67-15d8-4125-99f1-bb3a18655fe9 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:55.428071 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 264/532] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:09:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:55.429730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7ed67-15d8-4125-99f1-bb3a18655fe9 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:55.491352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc7378-777d-4ccb-ba0a-3ccdda042dfa None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:55.540451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7ed67-15d8-4125-99f1-bb3a18655fe9 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:55.540637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97158fe-5360-43e2-afcf-6a0fd11fb5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:55.541202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97158fe-5360-43e2-afcf-6a0fd11fb5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:55.541478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97158fe-5360-43e2-afcf-6a0fd11fb5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:55.541748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97158fe-5360-43e2-afcf-6a0fd11fb5d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:55.549197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:55.586018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a717a-5d7b-4d11-bcee-46a1cb35bed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:55.586637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a717a-5d7b-4d11-bcee-46a1cb35bed9 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:55.586918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a717a-5d7b-4d11-bcee-46a1cb35bed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:55.587191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a717a-5d7b-4d11-bcee-46a1cb35bed9 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:55.588542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a717a-5d7b-4d11-bcee-46a1cb35bed9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:55.588851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a717a-5d7b-4d11-bcee-46a1cb35bed9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:55.591227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a717a-5d7b-4d11-bcee-46a1cb35bed9 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:55.607051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 265/533] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:09:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:55.607776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a717a-5d7b-4d11-bcee-46a1cb35bed9 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:55.631030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97158fe-5360-43e2-afcf-6a0fd11fb5d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:55.632540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97158fe-5360-43e2-afcf-6a0fd11fb5d9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:55.681623 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:55.725322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a717a-5d7b-4d11-bcee-46a1cb35bed9 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:55.746586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 269/534] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:55.752587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97158fe-5360-43e2-afcf-6a0fd11fb5d9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:55.765099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0e65d-e736-43bb-9594-9447e4780818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:55.765983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0e65d-e736-43bb-9594-9447e4780818 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:55.766461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0e65d-e736-43bb-9594-9447e4780818 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:55.766932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0e65d-e736-43bb-9594-9447e4780818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:55.768107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0e65d-e736-43bb-9594-9447e4780818 None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=8f33f1f3d1ca4436887c2346bd9fa149) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:55.768666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0e65d-e736-43bb-9594-9447e4780818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:55.770307 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0e65d-e736-43bb-9594-9447e4780818 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:55.776701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 266/535] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:55 2022] GET /identity/v3/role_assignments?user.id=8f33f1f3d1ca4436887c2346bd9fa149&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:55.784834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0e65d-e736-43bb-9594-9447e4780818 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:55.856128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:55.860220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97158fe-5360-43e2-afcf-6a0fd11fb5d9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:55.896640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cddf5-d71f-4ab9-a23f-1924102f6be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:55.897259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cddf5-d71f-4ab9-a23f-1924102f6be7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:55.897538 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cddf5-d71f-4ab9-a23f-1924102f6be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:55.897807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cddf5-d71f-4ab9-a23f-1924102f6be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:55.898543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0cddf5-d71f-4ab9-a23f-1924102f6be7 None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=237bbeebe2a949aba444353c1d2ee04f) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:55.898849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0cddf5-d71f-4ab9-a23f-1924102f6be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:55.900315 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0cddf5-d71f-4ab9-a23f-1924102f6be7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:55.906587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 270/536] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:55 2022] GET /identity/v3/role_assignments?user.id=237bbeebe2a949aba444353c1d2ee04f&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:55.992151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0e65d-e736-43bb-9594-9447e4780818 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:56.056416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980f0bb-5eda-449e-905f-7b559e75dbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:56.057091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980f0bb-5eda-449e-905f-7b559e75dbdd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:56.057379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980f0bb-5eda-449e-905f-7b559e75dbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:56.057651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980f0bb-5eda-449e-905f-7b559e75dbdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:56.071585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980f0bb-5eda-449e-905f-7b559e75dbdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:56.072098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980f0bb-5eda-449e-905f-7b559e75dbdd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-3980f0bb-5eda-449e-905f-7b559e75dbdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:56.094822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:56.095997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:56.097041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:56.097041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:56.097041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:56.098527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 267/537] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:56.109275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cddf5-d71f-4ab9-a23f-1924102f6be7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:56.119915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:56.233186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cddf5-d71f-4ab9-a23f-1924102f6be7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:56.267297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4307b76-f5f0-455c-b702-e7efda97cf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:56.267947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4307b76-f5f0-455c-b702-e7efda97cf7e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:56.268225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4307b76-f5f0-455c-b702-e7efda97cf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:56.268557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4307b76-f5f0-455c-b702-e7efda97cf7e None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:56.269203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4307b76-f5f0-455c-b702-e7efda97cf7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:56.269503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4307b76-f5f0-455c-b702-e7efda97cf7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:56.270848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4307b76-f5f0-455c-b702-e7efda97cf7e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:56.287824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 271/538] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:56.289232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980f0bb-5eda-449e-905f-7b559e75dbdd None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:56.402026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:56.421172 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980f0bb-5eda-449e-905f-7b559e75dbdd None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:56.461154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29870d21-168c-463f-8e43-38285b649d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:56.461843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29870d21-168c-463f-8e43-38285b649d47 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:56.462095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29870d21-168c-463f-8e43-38285b649d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:56.462339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29870d21-168c-463f-8e43-38285b649d47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:56.476557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29870d21-168c-463f-8e43-38285b649d47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:56.478028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29870d21-168c-463f-8e43-38285b649d47 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-29870d21-168c-463f-8e43-38285b649d47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:09:56.494196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:56.495076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:56.496618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:09:56.496618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:09:56.496618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:09:56.496618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:09:56.496618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:09:56.496618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:09:56.496618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:09:56.496618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:56.496618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 268/539] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:09:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:56.501067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4307b76-f5f0-455c-b702-e7efda97cf7e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:56.577531 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:56.617103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4307b76-f5f0-455c-b702-e7efda97cf7e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:56.652856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4076fc6-0b1a-449c-9abe-675bcf4eb382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:56.653483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4076fc6-0b1a-449c-9abe-675bcf4eb382 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:56.653788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4076fc6-0b1a-449c-9abe-675bcf4eb382 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:56.654121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4076fc6-0b1a-449c-9abe-675bcf4eb382 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:56.655588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4076fc6-0b1a-449c-9abe-675bcf4eb382 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:56.655961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4076fc6-0b1a-449c-9abe-675bcf4eb382 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:56.657447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4076fc6-0b1a-449c-9abe-675bcf4eb382 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:56.676060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 272/540] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:09:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:56.681960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29870d21-168c-463f-8e43-38285b649d47 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:56.775713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9211cf-0108-4a59-a89a-87dc0789aea9 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:56.776082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9211cf-0108-4a59-a89a-87dc0789aea9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:56.776425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9211cf-0108-4a59-a89a-87dc0789aea9 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:56.777826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 273/541] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:09:56.784146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec67294-a882-4584-af32-6098c1d3ec38 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:56.784429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec67294-a882-4584-af32-6098c1d3ec38 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:56.784702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec67294-a882-4584-af32-6098c1d3ec38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:56.817744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29870d21-168c-463f-8e43-38285b649d47 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:56.819956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0ec67294-a882-4584-af32-6098c1d3ec38 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:56.821134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0ec67294-a882-4584-af32-6098c1d3ec38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:56.891419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050985cc-06e1-41a4-9774-4c23563b04a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:56.892051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050985cc-06e1-41a4-9774-4c23563b04a7 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:56.892331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050985cc-06e1-41a4-9774-4c23563b04a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:56.892604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050985cc-06e1-41a4-9774-4c23563b04a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:56.893326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050985cc-06e1-41a4-9774-4c23563b04a7 None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=9a89a2e6aeb4495a84a61bd82b02bfc4) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:56.893630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050985cc-06e1-41a4-9774-4c23563b04a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:56.895058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050985cc-06e1-41a4-9774-4c23563b04a7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:56.907268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 269/542] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:09:56 2022] GET /identity/v3/role_assignments?user.id=9a89a2e6aeb4495a84a61bd82b02bfc4&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:56.950250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 274/543] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:57.071289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5541a-3e85-4c34-9d3b-b86addc7fcb0 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:57.071596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5541a-3e85-4c34-9d3b-b86addc7fcb0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:57.071881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5541a-3e85-4c34-9d3b-b86addc7fcb0 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:57.073485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 270/544] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:57.079746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b64609-3c2e-4ddd-9c79-eac13d79b8f6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:57.080197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b64609-3c2e-4ddd-9c79-eac13d79b8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:57.080626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b64609-3c2e-4ddd-9c79-eac13d79b8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:57.100770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f0b64609-3c2e-4ddd-9c79-eac13d79b8f6 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:57.102197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f0b64609-3c2e-4ddd-9c79-eac13d79b8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:57.151821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:57.175801 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 275/545] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:09:57.403127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5541a-3e85-4c34-9d3b-b86addc7fcb0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:57.412236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:57.523082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5541a-3e85-4c34-9d3b-b86addc7fcb0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:57.573346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4774f-340b-43ba-9b3d-1f6cf735fd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:57.573988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4774f-340b-43ba-9b3d-1f6cf735fd7e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:57.574271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4774f-340b-43ba-9b3d-1f6cf735fd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:57.574550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4774f-340b-43ba-9b3d-1f6cf735fd7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:57.582381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b64609-3c2e-4ddd-9c79-eac13d79b8f6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:57.603009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4774f-340b-43ba-9b3d-1f6cf735fd7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-cfb4774f-340b-43ba-9b3d-1f6cf735fd7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:57.609357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:57.610264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:57.611090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:57.618411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 271/546] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:57.624026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4774f-340b-43ba-9b3d-1f6cf735fd7e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:57.755838 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b64609-3c2e-4ddd-9c79-eac13d79b8f6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:57.759448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:57.823591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477645b7-43cd-4a81-81ef-2bbdb6db6c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:57.824141 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477645b7-43cd-4a81-81ef-2bbdb6db6c4a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:57.824415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477645b7-43cd-4a81-81ef-2bbdb6db6c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:57.824682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477645b7-43cd-4a81-81ef-2bbdb6db6c4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:57.829976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4774f-340b-43ba-9b3d-1f6cf735fd7e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:57.842796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477645b7-43cd-4a81-81ef-2bbdb6db6c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-477645b7-43cd-4a81-81ef-2bbdb6db6c4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:57.847652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:57.848536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:57.849415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 276/547] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:57.856625 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477645b7-43cd-4a81-81ef-2bbdb6db6c4a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:57.867921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3759c7-9a99-4a01-9ebf-853dca44c569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:57.868632 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3759c7-9a99-4a01-9ebf-853dca44c569 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:57.868908 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3759c7-9a99-4a01-9ebf-853dca44c569 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:57.869225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3759c7-9a99-4a01-9ebf-853dca44c569 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:57.869887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3759c7-9a99-4a01-9ebf-853dca44c569 None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:57.870187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3759c7-9a99-4a01-9ebf-853dca44c569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:57.871599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3759c7-9a99-4a01-9ebf-853dca44c569 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:57.884809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 272/548] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:09:57.886021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3759c7-9a99-4a01-9ebf-853dca44c569 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:57.939893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:58.056254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477645b7-43cd-4a81-81ef-2bbdb6db6c4a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:58.082271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3759c7-9a99-4a01-9ebf-853dca44c569 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:58.117008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:58.118386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e927238-cc1a-48ab-b69d-76e9fdcefb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:58.119024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e927238-cc1a-48ab-b69d-76e9fdcefb5f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.119304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e927238-cc1a-48ab-b69d-76e9fdcefb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.119603 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e927238-cc1a-48ab-b69d-76e9fdcefb5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.137065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d17a45-6ac3-4dce-8830-5b6022ad9e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:58.137676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d17a45-6ac3-4dce-8830-5b6022ad9e2f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.137949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d17a45-6ac3-4dce-8830-5b6022ad9e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.138218 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d17a45-6ac3-4dce-8830-5b6022ad9e2f None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.139407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d17a45-6ac3-4dce-8830-5b6022ad9e2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:58.142061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d17a45-6ac3-4dce-8830-5b6022ad9e2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:58.145062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e927238-cc1a-48ab-b69d-76e9fdcefb5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:58.146896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d17a45-6ac3-4dce-8830-5b6022ad9e2f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:58.147825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e927238-cc1a-48ab-b69d-76e9fdcefb5f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:58.167173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 277/549] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:58.171237 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d17a45-6ac3-4dce-8830-5b6022ad9e2f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-1e927238-cc1a-48ab-b69d-76e9fdcefb5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:58.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:58.175890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:58.177995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:58.177995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:58.177995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:58.177995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 273/550] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:58.180416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e927238-cc1a-48ab-b69d-76e9fdcefb5f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:58.314867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e927238-cc1a-48ab-b69d-76e9fdcefb5f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:58.316436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d17a45-6ac3-4dce-8830-5b6022ad9e2f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:58.318712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:58.350068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635419f-b7e7-4540-8281-9e60244096dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:58.350623 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635419f-b7e7-4540-8281-9e60244096dd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.350932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635419f-b7e7-4540-8281-9e60244096dd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.351129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5c1c9-3ff9-406d-83d3-ecf9c91d82c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:58.351242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635419f-b7e7-4540-8281-9e60244096dd None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.351777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5c1c9-3ff9-406d-83d3-ecf9c91d82c6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.351961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635419f-b7e7-4540-8281-9e60244096dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:58.352043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5c1c9-3ff9-406d-83d3-ecf9c91d82c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.352203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635419f-b7e7-4540-8281-9e60244096dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:58.352296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5c1c9-3ff9-406d-83d3-ecf9c91d82c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.353602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635419f-b7e7-4540-8281-9e60244096dd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:58.365323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5c1c9-3ff9-406d-83d3-ecf9c91d82c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:58.365449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 274/551] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:58.366313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8fb40-976d-4b19-898b-0cb97dd84bd6 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.366597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8fb40-976d-4b19-898b-0cb97dd84bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.366755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5c1c9-3ff9-406d-83d3-ecf9c91d82c6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:58.366860 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8fb40-976d-4b19-898b-0cb97dd84bd6 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.379580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 275/552] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:58.379580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49942aa2-dee9-403e-81ff-8b784fa123f5 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.379580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49942aa2-dee9-403e-81ff-8b784fa123f5 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.379580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49942aa2-dee9-403e-81ff-8b784fa123f5 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.379580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 276/553] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:58.379580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49942aa2-dee9-403e-81ff-8b784fa123f5 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-0fd5c1c9-3ff9-406d-83d3-ecf9c91d82c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:58.386416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:58.387349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:58.388618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:58.388618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:58.388618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:58.388618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 278/554] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:09:58.389644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8b539-5a19-479b-9d49-a41d3b98cdbe None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.389923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8b539-5a19-479b-9d49-a41d3b98cdbe None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.390196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8b539-5a19-479b-9d49-a41d3b98cdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.413033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6bc8b539-5a19-479b-9d49-a41d3b98cdbe None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:58.414555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6bc8b539-5a19-479b-9d49-a41d3b98cdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:58.427402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:58.489017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49942aa2-dee9-403e-81ff-8b784fa123f5 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:58.511561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 279/555] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:58.511561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984cdc3-8e05-4447-ad44-d3a89cb4234a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.511561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984cdc3-8e05-4447-ad44-d3a89cb4234a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.511561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984cdc3-8e05-4447-ad44-d3a89cb4234a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.527308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1984cdc3-8e05-4447-ad44-d3a89cb4234a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:58.528640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1984cdc3-8e05-4447-ad44-d3a89cb4234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:58.545516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c49e82-092b-411e-924a-64fe47b2b16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:58.546353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c49e82-092b-411e-924a-64fe47b2b16d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.546702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c49e82-092b-411e-924a-64fe47b2b16d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.547192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c49e82-092b-411e-924a-64fe47b2b16d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.555737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c49e82-092b-411e-924a-64fe47b2b16d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:58.557453 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c49e82-092b-411e-924a-64fe47b2b16d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-35c49e82-092b-411e-924a-64fe47b2b16d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:58.573569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:58.574529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:58.575461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:58.575461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:58.575461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:58.575461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:58.575461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:58.575461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:58.575461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:58.575461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:58.575461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:09:58.575461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:58.581642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 277/556] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:09:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:58.582626 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c49e82-092b-411e-924a-64fe47b2b16d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:58.627770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 280/557] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:58.629595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984cdc3-8e05-4447-ad44-d3a89cb4234a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:58.663845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:58.753745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:58.757100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c49e82-092b-411e-924a-64fe47b2b16d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:58.812214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984cdc3-8e05-4447-ad44-d3a89cb4234a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:58.815874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bb462-347c-4cd2-bbfc-015dd1114abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:58.816489 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bb462-347c-4cd2-bbfc-015dd1114abf None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.816773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bb462-347c-4cd2-bbfc-015dd1114abf None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.817051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bb462-347c-4cd2-bbfc-015dd1114abf None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.817700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671bb462-347c-4cd2-bbfc-015dd1114abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:58.818007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671bb462-347c-4cd2-bbfc-015dd1114abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:58.819406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671bb462-347c-4cd2-bbfc-015dd1114abf None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:58.832182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 278/558] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:58.832824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bb462-347c-4cd2-bbfc-015dd1114abf None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:58.848614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0e221-7e35-46c1-a19a-f57316ee1cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:58.849219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0e221-7e35-46c1-a19a-f57316ee1cce None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.849490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0e221-7e35-46c1-a19a-f57316ee1cce None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.849861 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0e221-7e35-46c1-a19a-f57316ee1cce None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:58.850690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0e221-7e35-46c1-a19a-f57316ee1cce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:58.850992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0e221-7e35-46c1-a19a-f57316ee1cce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:58.852441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0e221-7e35-46c1-a19a-f57316ee1cce None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:58.892544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 281/559] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Sep 5 12:09:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:58.893318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0e221-7e35-46c1-a19a-f57316ee1cce None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:58.954034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bb462-347c-4cd2-bbfc-015dd1114abf None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:58.983520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:58.992386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543be29-1df3-4153-9bf8-7f5b977e6b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:58.997284 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543be29-1df3-4153-9bf8-7f5b977e6b3a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:58.997284 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543be29-1df3-4153-9bf8-7f5b977e6b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:58.997284 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543be29-1df3-4153-9bf8-7f5b977e6b3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.004261 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543be29-1df3-4153-9bf8-7f5b977e6b3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:59.008104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0e221-7e35-46c1-a19a-f57316ee1cce None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-4543be29-1df3-4153-9bf8-7f5b977e6b3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:59.009526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:59.010413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 279/560] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:59.011991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543be29-1df3-4153-9bf8-7f5b977e6b3a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.040091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:59.052092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38aac1-0549-4ecb-9e67-55edd4dbcb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:59.052732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38aac1-0549-4ecb-9e67-55edd4dbcb18 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.053175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38aac1-0549-4ecb-9e67-55edd4dbcb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.053448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38aac1-0549-4ecb-9e67-55edd4dbcb18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.061184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38aac1-0549-4ecb-9e67-55edd4dbcb18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:59.062620 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38aac1-0549-4ecb-9e67-55edd4dbcb18 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-fe38aac1-0549-4ecb-9e67-55edd4dbcb18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:09:59.074834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:59.075935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:59.077396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 282/561] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Sep 5 12:09:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:59.078001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38aac1-0549-4ecb-9e67-55edd4dbcb18 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.123981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543be29-1df3-4153-9bf8-7f5b977e6b3a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.157201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4816d-47d6-45ab-ad09-bd46eb18d72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:59.157728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4816d-47d6-45ab-ad09-bd46eb18d72a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.158002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4816d-47d6-45ab-ad09-bd46eb18d72a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.158268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4816d-47d6-45ab-ad09-bd46eb18d72a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.177918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38aac1-0549-4ecb-9e67-55edd4dbcb18 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.210634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e69e8-8106-47fa-babd-7406affb4645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:59.211261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e69e8-8106-47fa-babd-7406affb4645 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.211568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e69e8-8106-47fa-babd-7406affb4645 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.211858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e69e8-8106-47fa-babd-7406affb4645 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.213934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:59.223513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e69e8-8106-47fa-babd-7406affb4645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:09:59.224090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4816d-47d6-45ab-ad09-bd46eb18d72a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:59.225548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4816d-47d6-45ab-ad09-bd46eb18d72a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-620e69e8-8106-47fa-babd-7406affb4645 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:09:59.228324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:59.229214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:59.230053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 283/562] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:59.230770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e69e8-8106-47fa-babd-7406affb4645 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.247427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:59.324880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 280/563] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:09:59.326106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bd991-88ab-4a73-9210-744a5ccf954e None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.326415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bd991-88ab-4a73-9210-744a5ccf954e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.326690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bd991-88ab-4a73-9210-744a5ccf954e None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.328095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 281/564] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:09:59.329012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e69e8-8106-47fa-babd-7406affb4645 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.332519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bd991-88ab-4a73-9210-744a5ccf954e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.362652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5a358-fbb3-456d-b974-b609635f3c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:59.363252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5a358-fbb3-456d-b974-b609635f3c53 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.363527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5a358-fbb3-456d-b974-b609635f3c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.363827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5a358-fbb3-456d-b974-b609635f3c53 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.364484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5a358-fbb3-456d-b974-b609635f3c53 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:59.364786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5a358-fbb3-456d-b974-b609635f3c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:59.366125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5a358-fbb3-456d-b974-b609635f3c53 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:59.374520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 284/565] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:59.375287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5a358-fbb3-456d-b974-b609635f3c53 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.432622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:59.438122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bd991-88ab-4a73-9210-744a5ccf954e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.474735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5a358-fbb3-456d-b974-b609635f3c53 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.496689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:59.510652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b814fe9-566b-4e38-8dd6-3026284cfed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:59.511043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d8461-62c0-43df-bc9d-2a14e8432c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:59.511245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b814fe9-566b-4e38-8dd6-3026284cfed0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.511522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b814fe9-566b-4e38-8dd6-3026284cfed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.511675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d8461-62c0-43df-bc9d-2a14e8432c1d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.511822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b814fe9-566b-4e38-8dd6-3026284cfed0 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.511988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d8461-62c0-43df-bc9d-2a14e8432c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.512264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d8461-62c0-43df-bc9d-2a14e8432c1d None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.512531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b814fe9-566b-4e38-8dd6-3026284cfed0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:59.512860 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b814fe9-566b-4e38-8dd6-3026284cfed0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:59.513012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d8461-62c0-43df-bc9d-2a14e8432c1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:09:59.513208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d8461-62c0-43df-bc9d-2a14e8432c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:59.514303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b814fe9-566b-4e38-8dd6-3026284cfed0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:59.514586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d8461-62c0-43df-bc9d-2a14e8432c1d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:59.522583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 285/566] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:59.523364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d8461-62c0-43df-bc9d-2a14e8432c1d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.532444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 282/567] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:09:59.533578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feddf32-948f-43aa-ac4c-ce07dda442e9 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.534136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feddf32-948f-43aa-ac4c-ce07dda442e9 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.534472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feddf32-948f-43aa-ac4c-ce07dda442e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.552156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8feddf32-948f-43aa-ac4c-ce07dda442e9 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:09:59.553452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8feddf32-948f-43aa-ac4c-ce07dda442e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:09:59.618829 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 283/568] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:09:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:09:59.620430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feddf32-948f-43aa-ac4c-ce07dda442e9 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.622673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d8461-62c0-43df-bc9d-2a14e8432c1d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.640126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:59.657571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a63e0a-8b35-4329-9e82-ab13051a0f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:59.658201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a63e0a-8b35-4329-9e82-ab13051a0f1c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.658478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a63e0a-8b35-4329-9e82-ab13051a0f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.658750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a63e0a-8b35-4329-9e82-ab13051a0f1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.670549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a63e0a-8b35-4329-9e82-ab13051a0f1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:59.672110 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a63e0a-8b35-4329-9e82-ab13051a0f1c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-44a63e0a-8b35-4329-9e82-ab13051a0f1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:59.686648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:59.687613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:59.688522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:59.688522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:59.688522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:59.688522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 286/569] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:59.688522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a63e0a-8b35-4329-9e82-ab13051a0f1c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.744041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feddf32-948f-43aa-ac4c-ce07dda442e9 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.753777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:59.777819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f9774-b76d-4681-b71f-7a20ce7df2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:59.778668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f9774-b76d-4681-b71f-7a20ce7df2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.778945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f9774-b76d-4681-b71f-7a20ce7df2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.779215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f9774-b76d-4681-b71f-7a20ce7df2b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.789050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f9774-b76d-4681-b71f-7a20ce7df2b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:59.790597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f9774-b76d-4681-b71f-7a20ce7df2b0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:59.796951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a63e0a-8b35-4329-9e82-ab13051a0f1c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-523f9774-b76d-4681-b71f-7a20ce7df2b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:59.806484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:59.807428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:59.808356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:59.808356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:59.808356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:59.808356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 284/570] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:59.808356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f9774-b76d-4681-b71f-7a20ce7df2b0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.836002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e856934-6f26-4984-9449-e4b7465e341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:59.836707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e856934-6f26-4984-9449-e4b7465e341d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.836921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e856934-6f26-4984-9449-e4b7465e341d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.837172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e856934-6f26-4984-9449-e4b7465e341d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.849496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e856934-6f26-4984-9449-e4b7465e341d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:59.850917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e856934-6f26-4984-9449-e4b7465e341d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-4e856934-6f26-4984-9449-e4b7465e341d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:59.865727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:59.871652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:59.872653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:59.872653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:59.872653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:09:59.872653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 287/571] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:59.872653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e856934-6f26-4984-9449-e4b7465e341d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.896908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:09:59.921423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f9774-b76d-4681-b71f-7a20ce7df2b0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.953898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31654b9c-5385-4d94-a95a-165777621650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:09:59.954450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31654b9c-5385-4d94-a95a-165777621650 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:09:59.954725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31654b9c-5385-4d94-a95a-165777621650 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:09:59.954996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31654b9c-5385-4d94-a95a-165777621650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:09:59.965967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31654b9c-5385-4d94-a95a-165777621650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:09:59.967931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31654b9c-5385-4d94-a95a-165777621650 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:09:59.971382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e856934-6f26-4984-9449-e4b7465e341d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-31654b9c-5385-4d94-a95a-165777621650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:09:59.982032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:09:59.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:09:59.984618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:09:59.984618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:09:59.984618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:09:59.984618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 285/572] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:09:59.984618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31654b9c-5385-4d94-a95a-165777621650 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:09:59.998800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:00.009919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8e941-2a1b-4602-81f8-644fddf47be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.024290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8e941-2a1b-4602-81f8-644fddf47be8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.024290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8e941-2a1b-4602-81f8-644fddf47be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.024290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8e941-2a1b-4602-81f8-644fddf47be8 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.024290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8e941-2a1b-4602-81f8-644fddf47be8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:00.024290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8e941-2a1b-4602-81f8-644fddf47be8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.024290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8e941-2a1b-4602-81f8-644fddf47be8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.024836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 288/573] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:00.025627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8e941-2a1b-4602-81f8-644fddf47be8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.084885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31654b9c-5385-4d94-a95a-165777621650 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.107606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:00.118581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08f17d-0f08-4a76-8c60-37170408ee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.119174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f17d-0f08-4a76-8c60-37170408ee9b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.119450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f17d-0f08-4a76-8c60-37170408ee9b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.121502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f17d-0f08-4a76-8c60-37170408ee9b None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.121502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08f17d-0f08-4a76-8c60-37170408ee9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:00.121502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08f17d-0f08-4a76-8c60-37170408ee9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.122018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08f17d-0f08-4a76-8c60-37170408ee9b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.125494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8e941-2a1b-4602-81f8-644fddf47be8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.132918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 286/574] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:00.133692 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08f17d-0f08-4a76-8c60-37170408ee9b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.161626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad77731-d1f5-4538-ae9b-7cbdd01d677d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.162233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad77731-d1f5-4538-ae9b-7cbdd01d677d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.162523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad77731-d1f5-4538-ae9b-7cbdd01d677d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.162798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad77731-d1f5-4538-ae9b-7cbdd01d677d None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.163434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad77731-d1f5-4538-ae9b-7cbdd01d677d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:00.163767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad77731-d1f5-4538-ae9b-7cbdd01d677d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.165129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad77731-d1f5-4538-ae9b-7cbdd01d677d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.176747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 289/575] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:00.177381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad77731-d1f5-4538-ae9b-7cbdd01d677d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.205753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:00.235694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08f17d-0f08-4a76-8c60-37170408ee9b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.268993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b08230-4ec4-4f1b-a0a1-df7f62d54d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.269613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b08230-4ec4-4f1b-a0a1-df7f62d54d76 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.269893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b08230-4ec4-4f1b-a0a1-df7f62d54d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.270164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b08230-4ec4-4f1b-a0a1-df7f62d54d76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.278325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad77731-d1f5-4538-ae9b-7cbdd01d677d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.280374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b08230-4ec4-4f1b-a0a1-df7f62d54d76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-12b08230-4ec4-4f1b-a0a1-df7f62d54d76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:00.285562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:00.286467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:00.287297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:00.291377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 287/576] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:00.292207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b08230-4ec4-4f1b-a0a1-df7f62d54d76 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.311746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71262a3-5f4d-4f9c-b0dc-ed0697edb995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.312343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71262a3-5f4d-4f9c-b0dc-ed0697edb995 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.312622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71262a3-5f4d-4f9c-b0dc-ed0697edb995 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.312894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71262a3-5f4d-4f9c-b0dc-ed0697edb995 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.313529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71262a3-5f4d-4f9c-b0dc-ed0697edb995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:00.313827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71262a3-5f4d-4f9c-b0dc-ed0697edb995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.315156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71262a3-5f4d-4f9c-b0dc-ed0697edb995 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.327097 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 290/577] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:00.327894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71262a3-5f4d-4f9c-b0dc-ed0697edb995 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.330325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:00.389348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b08230-4ec4-4f1b-a0a1-df7f62d54d76 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.417921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:00.423234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38ec54-a24d-4157-91df-0db976e8531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.423948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38ec54-a24d-4157-91df-0db976e8531c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.424230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38ec54-a24d-4157-91df-0db976e8531c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.424506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38ec54-a24d-4157-91df-0db976e8531c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.426454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71262a3-5f4d-4f9c-b0dc-ed0697edb995 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.431035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38ec54-a24d-4157-91df-0db976e8531c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.432521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38ec54-a24d-4157-91df-0db976e8531c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-4f38ec54-a24d-4157-91df-0db976e8531c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:00.443971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:00.445036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:00.445955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:00.445955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:00.445955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:00.445955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:00.445955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:00.445955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:00.445955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:00.445955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:00.445955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 288/578] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:00.446445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38ec54-a24d-4157-91df-0db976e8531c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.460825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac19fcd-7f9f-42bd-b3c2-c6d94948459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.461402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac19fcd-7f9f-42bd-b3c2-c6d94948459c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.461677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac19fcd-7f9f-42bd-b3c2-c6d94948459c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.461951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac19fcd-7f9f-42bd-b3c2-c6d94948459c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.469131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac19fcd-7f9f-42bd-b3c2-c6d94948459c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.470542 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac19fcd-7f9f-42bd-b3c2-c6d94948459c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-aac19fcd-7f9f-42bd-b3c2-c6d94948459c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:00.481917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:00.483348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 291/579] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:00.484506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac19fcd-7f9f-42bd-b3c2-c6d94948459c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.544783 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:00.546737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38ec54-a24d-4157-91df-0db976e8531c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.580040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514b797-d9cb-4832-8265-79388179cc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.580735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514b797-d9cb-4832-8265-79388179cc7f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.580937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514b797-d9cb-4832-8265-79388179cc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.581205 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514b797-d9cb-4832-8265-79388179cc7f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.583422 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac19fcd-7f9f-42bd-b3c2-c6d94948459c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.588620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514b797-d9cb-4832-8265-79388179cc7f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.590042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514b797-d9cb-4832-8265-79388179cc7f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-5514b797-d9cb-4832-8265-79388179cc7f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:00.601362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:00.602434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 289/580] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:00.608777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514b797-d9cb-4832-8265-79388179cc7f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.623317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd039f79-5f42-45cf-a16f-1175f807c751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.623972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd039f79-5f42-45cf-a16f-1175f807c751 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.624251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd039f79-5f42-45cf-a16f-1175f807c751 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.624520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd039f79-5f42-45cf-a16f-1175f807c751 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.625172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd039f79-5f42-45cf-a16f-1175f807c751 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:00.625471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd039f79-5f42-45cf-a16f-1175f807c751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.626831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd039f79-5f42-45cf-a16f-1175f807c751 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.634622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 292/581] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:00.635400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd039f79-5f42-45cf-a16f-1175f807c751 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.657632 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:00.704106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514b797-d9cb-4832-8265-79388179cc7f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.731837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd039f79-5f42-45cf-a16f-1175f807c751 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.738821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08603c0-bb77-4672-9855-e5255ede8d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.739447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08603c0-bb77-4672-9855-e5255ede8d2a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.739759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08603c0-bb77-4672-9855-e5255ede8d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.740032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08603c0-bb77-4672-9855-e5255ede8d2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.746497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08603c0-bb77-4672-9855-e5255ede8d2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.747974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08603c0-bb77-4672-9855-e5255ede8d2a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.754169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-b08603c0-bb77-4672-9855-e5255ede8d2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:00.758916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:00.759987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:00.761290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:00.761290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:00.761290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:00.761290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:00.761290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:00.761290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:00.761290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:00.761290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:00.761290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 290/582] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:00.762712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08603c0-bb77-4672-9855-e5255ede8d2a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.765009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131a947-b234-4e42-b309-1f493a6596ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.765615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131a947-b234-4e42-b309-1f493a6596ae None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.765887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131a947-b234-4e42-b309-1f493a6596ae None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.766159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131a947-b234-4e42-b309-1f493a6596ae None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.766849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131a947-b234-4e42-b309-1f493a6596ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:00.769034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131a947-b234-4e42-b309-1f493a6596ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.769034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131a947-b234-4e42-b309-1f493a6596ae None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.781233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 293/583] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:00.782017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131a947-b234-4e42-b309-1f493a6596ae None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.863730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08603c0-bb77-4672-9855-e5255ede8d2a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.876009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:00.882891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131a947-b234-4e42-b309-1f493a6596ae None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:00.896390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7aa99-9bbc-4de9-9975-85ff5e6c6bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.897215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7aa99-9bbc-4de9-9975-85ff5e6c6bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.897497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7aa99-9bbc-4de9-9975-85ff5e6c6bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.897933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7aa99-9bbc-4de9-9975-85ff5e6c6bb7 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.898915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7aa99-9bbc-4de9-9975-85ff5e6c6bb7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:00.899220 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7aa99-9bbc-4de9-9975-85ff5e6c6bb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.900687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7aa99-9bbc-4de9-9975-85ff5e6c6bb7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.917697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1f3e0-1457-439b-b598-00b38c01ed94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:00.922786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 291/584] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:00.922942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1f3e0-1457-439b-b598-00b38c01ed94 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:00.922942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1f3e0-1457-439b-b598-00b38c01ed94 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:00.922942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1f3e0-1457-439b-b598-00b38c01ed94 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:00.923179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7aa99-9bbc-4de9-9975-85ff5e6c6bb7 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.923277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1f3e0-1457-439b-b598-00b38c01ed94 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:00.923277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1f3e0-1457-439b-b598-00b38c01ed94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:00.923277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1f3e0-1457-439b-b598-00b38c01ed94 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:00.938589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 294/585] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:00.939403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1f3e0-1457-439b-b598-00b38c01ed94 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:00.980609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:01.031543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7aa99-9bbc-4de9-9975-85ff5e6c6bb7 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:01.037170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1f3e0-1457-439b-b598-00b38c01ed94 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:01.070382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1360580-de0a-4a81-94c5-e7efd9b92e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:01.071009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1360580-de0a-4a81-94c5-e7efd9b92e56 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:01.071290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1360580-de0a-4a81-94c5-e7efd9b92e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:01.071588 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1360580-de0a-4a81-94c5-e7efd9b92e56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:01.072111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90373a77-9901-479a-a429-a66e58439172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:01.072707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90373a77-9901-479a-a429-a66e58439172 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:01.072986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90373a77-9901-479a-a429-a66e58439172 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:01.073259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90373a77-9901-479a-a429-a66e58439172 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:01.073949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90373a77-9901-479a-a429-a66e58439172 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:01.074471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90373a77-9901-479a-a429-a66e58439172 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:01.075692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90373a77-9901-479a-a429-a66e58439172 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:01.082272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1360580-de0a-4a81-94c5-e7efd9b92e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:01.083736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1360580-de0a-4a81-94c5-e7efd9b92e56 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:01.087443 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 295/586] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:01.088512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90373a77-9901-479a-a429-a66e58439172 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-f1360580-de0a-4a81-94c5-e7efd9b92e56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:01.099412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:01.100390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:01.101263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:01.101263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:01.101263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:01.101263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 292/587] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:01.101263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1360580-de0a-4a81-94c5-e7efd9b92e56 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:01.125058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:01.189735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90373a77-9901-479a-a429-a66e58439172 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:01.191255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:01.198855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1360580-de0a-4a81-94c5-e7efd9b92e56 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:01.225123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c039f7-ba92-420d-a3db-5fefe437a6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:01.225725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c039f7-ba92-420d-a3db-5fefe437a6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:01.225998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c039f7-ba92-420d-a3db-5fefe437a6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:01.226269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c039f7-ba92-420d-a3db-5fefe437a6c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:01.227056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c039f7-ba92-420d-a3db-5fefe437a6c8 None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=b48aa41ff66b42329926bc42379d9800) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:01.227355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c039f7-ba92-420d-a3db-5fefe437a6c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:01.228870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c039f7-ba92-420d-a3db-5fefe437a6c8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:01.234298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 296/588] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:01 2022] GET /identity/v3/role_assignments?user.id=b48aa41ff66b42329926bc42379d9800&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:01.234790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb25a7-11d2-46ca-b5df-cb46ea65bdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:01.235071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c039f7-ba92-420d-a3db-5fefe437a6c8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:01.235428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb25a7-11d2-46ca-b5df-cb46ea65bdc2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:01.235724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb25a7-11d2-46ca-b5df-cb46ea65bdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:01.236001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb25a7-11d2-46ca-b5df-cb46ea65bdc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:01.249833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb25a7-11d2-46ca-b5df-cb46ea65bdc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:01.251354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb25a7-11d2-46ca-b5df-cb46ea65bdc2 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-00bb25a7-11d2-46ca-b5df-cb46ea65bdc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:01.269275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:01.270392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:01.271263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:01.271263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:01.271263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:01.271263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 293/589] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:01.271953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb25a7-11d2-46ca-b5df-cb46ea65bdc2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:01.357226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:01.361400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c039f7-ba92-420d-a3db-5fefe437a6c8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:01.385174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb25a7-11d2-46ca-b5df-cb46ea65bdc2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:01.398170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f14983-d70c-4b98-8015-5be1771b7622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:01.398799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f14983-d70c-4b98-8015-5be1771b7622 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:01.399074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f14983-d70c-4b98-8015-5be1771b7622 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:01.399342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f14983-d70c-4b98-8015-5be1771b7622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:01.411659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f14983-d70c-4b98-8015-5be1771b7622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:01.413093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f14983-d70c-4b98-8015-5be1771b7622 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:01.419987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6537d8d-ef76-4dbf-a1c4-2bb645fa066b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:01.420705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6537d8d-ef76-4dbf-a1c4-2bb645fa066b None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:01.420990 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6537d8d-ef76-4dbf-a1c4-2bb645fa066b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:01.421265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6537d8d-ef76-4dbf-a1c4-2bb645fa066b None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/7956a7748ec94a618eb466f5536220b0/roles/dd50550e774349d6af84f593eeeec10d` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-d9f14983-d70c-4b98-8015-5be1771b7622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:01.427883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:01.434148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:01.435289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:01.435289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:01.435289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:01.435289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 297/590] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:01.435289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f14983-d70c-4b98-8015-5be1771b7622 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:01.453358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:01.458408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6537d8d-ef76-4dbf-a1c4-2bb645fa066b None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=7956a7748ec94a618eb466f5536220b0, role_id=dd50550e774349d6af84f593eeeec10d)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:01.459908 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6537d8d-ef76-4dbf-a1c4-2bb645fa066b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:01.523566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 294/591] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:10:01 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/7956a7748ec94a618eb466f5536220b0/roles/dd50550e774349d6af84f593eeeec10d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:10:01.523566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6537d8d-ef76-4dbf-a1c4-2bb645fa066b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:01.553133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f14983-d70c-4b98-8015-5be1771b7622 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:01.610172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db65446-d793-4ffd-a3a2-41f2d3231967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:01.610756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db65446-d793-4ffd-a3a2-41f2d3231967 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:01.611042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db65446-d793-4ffd-a3a2-41f2d3231967 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:01.611316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db65446-d793-4ffd-a3a2-41f2d3231967 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:01.612577 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db65446-d793-4ffd-a3a2-41f2d3231967 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:01.612880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db65446-d793-4ffd-a3a2-41f2d3231967 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:01.615097 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db65446-d793-4ffd-a3a2-41f2d3231967 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:01.634716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 298/592] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:01.636460 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db65446-d793-4ffd-a3a2-41f2d3231967 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:01.647305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6537d8d-ef76-4dbf-a1c4-2bb645fa066b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:01.674531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:01.690286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30aa3-985e-4170-9063-6fcff223d47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:01.690982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30aa3-985e-4170-9063-6fcff223d47b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:01.694960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30aa3-985e-4170-9063-6fcff223d47b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:01.694960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30aa3-985e-4170-9063-6fcff223d47b None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:01.694960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30aa3-985e-4170-9063-6fcff223d47b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:01.694960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30aa3-985e-4170-9063-6fcff223d47b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:01.694960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30aa3-985e-4170-9063-6fcff223d47b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:01.708223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 295/593] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:01.711587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30aa3-985e-4170-9063-6fcff223d47b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:01.748538 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:01.775208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db65446-d793-4ffd-a3a2-41f2d3231967 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:01.819397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803ac07-010a-4a86-99fc-c1a5ac8ecb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:01.820371 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ac07-010a-4a86-99fc-c1a5ac8ecb90 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:01.820842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ac07-010a-4a86-99fc-c1a5ac8ecb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:01.821866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ac07-010a-4a86-99fc-c1a5ac8ecb90 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:01.823014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ac07-010a-4a86-99fc-c1a5ac8ecb90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:01.823519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ac07-010a-4a86-99fc-c1a5ac8ecb90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:01.826027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ac07-010a-4a86-99fc-c1a5ac8ecb90 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:01.835076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30aa3-985e-4170-9063-6fcff223d47b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:01.843165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 299/594] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:01.844442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803ac07-010a-4a86-99fc-c1a5ac8ecb90 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:01.878325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628a8ac-fc91-4906-b63c-f737f6592042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:01.879153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628a8ac-fc91-4906-b63c-f737f6592042 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:01.879437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628a8ac-fc91-4906-b63c-f737f6592042 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:01.879753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628a8ac-fc91-4906-b63c-f737f6592042 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:01.890163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628a8ac-fc91-4906-b63c-f737f6592042 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:01.892645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628a8ac-fc91-4906-b63c-f737f6592042 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-2628a8ac-fc91-4906-b63c-f737f6592042 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:01.909112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:01.910104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:01.911046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:01.911046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:01.911046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:01.911046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:01.911046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:01.911046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:01.911046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:01.911046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:01.911046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:10:01.911046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:01.915253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 296/595] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:01.916116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628a8ac-fc91-4906-b63c-f737f6592042 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:01.976252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:01.981790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803ac07-010a-4a86-99fc-c1a5ac8ecb90 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:02.026391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c05c71-455e-42a5-85ef-ea288d1b398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:02.027834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c05c71-455e-42a5-85ef-ea288d1b398f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:02.027976 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c05c71-455e-42a5-85ef-ea288d1b398f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:02.028230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c05c71-455e-42a5-85ef-ea288d1b398f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:02.037088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c05c71-455e-42a5-85ef-ea288d1b398f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:02.038563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c05c71-455e-42a5-85ef-ea288d1b398f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:02.040608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:02.043772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628a8ac-fc91-4906-b63c-f737f6592042 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-96c05c71-455e-42a5-85ef-ea288d1b398f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:02.050802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:02.060634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:02.062162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:02.062162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:02.062162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:02.062162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:02.062162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:02.062162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:02.062162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:02.062162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:02.062162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 300/596] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:02.062162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c05c71-455e-42a5-85ef-ea288d1b398f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:02.092051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a07332-d0b9-4c6f-bdf2-68c4ec6316c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:02.093601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07332-d0b9-4c6f-bdf2-68c4ec6316c4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:02.093948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07332-d0b9-4c6f-bdf2-68c4ec6316c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:02.094294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07332-d0b9-4c6f-bdf2-68c4ec6316c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:02.102941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a07332-d0b9-4c6f-bdf2-68c4ec6316c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:02.105326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a07332-d0b9-4c6f-bdf2-68c4ec6316c4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-33a07332-d0b9-4c6f-bdf2-68c4ec6316c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:02.119689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:02.120613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:02.121577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:02.121577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:02.121577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:02.121577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:02.121577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:02.121577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:02.121577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:02.121577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:02.125289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 297/597] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:02.126639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a07332-d0b9-4c6f-bdf2-68c4ec6316c4 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:02.183738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c05c71-455e-42a5-85ef-ea288d1b398f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:02.236276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1916a74-6fb6-4bb0-b644-5723d444b1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:02.237048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916a74-6fb6-4bb0-b644-5723d444b1ed None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:02.237482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916a74-6fb6-4bb0-b644-5723d444b1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:02.237845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916a74-6fb6-4bb0-b644-5723d444b1ed None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:02.238680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1916a74-6fb6-4bb0-b644-5723d444b1ed None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:02.239085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1916a74-6fb6-4bb0-b644-5723d444b1ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:02.240635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1916a74-6fb6-4bb0-b644-5723d444b1ed None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:02.251219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a07332-d0b9-4c6f-bdf2-68c4ec6316c4 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:02.253735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:02.259497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 301/598] 10.222.0.62 () {56 vars in 1139 bytes} [Mon Sep 5 12:10:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:02.265404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1916a74-6fb6-4bb0-b644-5723d444b1ed None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:02.289076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96871ff8-03e8-4a58-b3c0-e25de7df137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:02.289944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96871ff8-03e8-4a58-b3c0-e25de7df137f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:02.290398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96871ff8-03e8-4a58-b3c0-e25de7df137f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:02.290817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96871ff8-03e8-4a58-b3c0-e25de7df137f None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:02.291787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96871ff8-03e8-4a58-b3c0-e25de7df137f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:02.292330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96871ff8-03e8-4a58-b3c0-e25de7df137f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:02.294059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96871ff8-03e8-4a58-b3c0-e25de7df137f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:02.308079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 298/599] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:02.309164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96871ff8-03e8-4a58-b3c0-e25de7df137f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:02.325846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:02.373369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1916a74-6fb6-4bb0-b644-5723d444b1ed None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:02.408560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863deb6d-a0ae-47c5-917a-0fe372651266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:02.409353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863deb6d-a0ae-47c5-917a-0fe372651266 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:02.409729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863deb6d-a0ae-47c5-917a-0fe372651266 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:02.410101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863deb6d-a0ae-47c5-917a-0fe372651266 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:02.410918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863deb6d-a0ae-47c5-917a-0fe372651266 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:02.411421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863deb6d-a0ae-47c5-917a-0fe372651266 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:02.412975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863deb6d-a0ae-47c5-917a-0fe372651266 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:02.424815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96871ff8-03e8-4a58-b3c0-e25de7df137f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:02.430435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 302/600] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:02.430435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863deb6d-a0ae-47c5-917a-0fe372651266 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:02.467929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48669b87-03ae-4392-b0f5-09c4295f5e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:02.468795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48669b87-03ae-4392-b0f5-09c4295f5e2d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:02.469219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48669b87-03ae-4392-b0f5-09c4295f5e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:02.469744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48669b87-03ae-4392-b0f5-09c4295f5e2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:02.481712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48669b87-03ae-4392-b0f5-09c4295f5e2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:02.484044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48669b87-03ae-4392-b0f5-09c4295f5e2d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-48669b87-03ae-4392-b0f5-09c4295f5e2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:02.499763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:02.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:02.502001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:02.502001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:02.502001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:02.503702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 299/601] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:02.504668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48669b87-03ae-4392-b0f5-09c4295f5e2d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:02.524813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:02.538525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863deb6d-a0ae-47c5-917a-0fe372651266 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:02.555238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:02.574284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c6895-97d0-4c55-beda-31242177875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:02.575106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c6895-97d0-4c55-beda-31242177875c None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:02.575481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c6895-97d0-4c55-beda-31242177875c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:02.575907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c6895-97d0-4c55-beda-31242177875c None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/237bbeebe2a949aba444353c1d2ee04f/roles/dd50550e774349d6af84f593eeeec10d` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:02.594453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c6895-97d0-4c55-beda-31242177875c None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=237bbeebe2a949aba444353c1d2ee04f, role_id=dd50550e774349d6af84f593eeeec10d)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:02.596417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c6895-97d0-4c55-beda-31242177875c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:02.619521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48669b87-03ae-4392-b0f5-09c4295f5e2d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:02.656366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5097c-ebd6-49de-a6f6-dce3c84b2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:02.657153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5097c-ebd6-49de-a6f6-dce3c84b2e20 None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:02.657420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5097c-ebd6-49de-a6f6-dce3c84b2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:02.657696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5097c-ebd6-49de-a6f6-dce3c84b2e20 None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/8f33f1f3d1ca4436887c2346bd9fa149/roles/dd50550e774349d6af84f593eeeec10d` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:02.662043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 303/602] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:10:02 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/237bbeebe2a949aba444353c1d2ee04f/roles/dd50550e774349d6af84f593eeeec10d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:10:02.663135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c6895-97d0-4c55-beda-31242177875c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:02.676795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5097c-ebd6-49de-a6f6-dce3c84b2e20 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=8f33f1f3d1ca4436887c2346bd9fa149, role_id=dd50550e774349d6af84f593eeeec10d)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:02.678928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5097c-ebd6-49de-a6f6-dce3c84b2e20 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:02.791933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 300/603] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:10:02 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/8f33f1f3d1ca4436887c2346bd9fa149/roles/dd50550e774349d6af84f593eeeec10d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:10:02.793144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c6895-97d0-4c55-beda-31242177875c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:02.835660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dc698-beed-4df2-be30-7ff276a5b13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:02.836719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dc698-beed-4df2-be30-7ff276a5b13d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:02.837298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dc698-beed-4df2-be30-7ff276a5b13d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:02.837769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dc698-beed-4df2-be30-7ff276a5b13d None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:02.838934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dc698-beed-4df2-be30-7ff276a5b13d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:02.839471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dc698-beed-4df2-be30-7ff276a5b13d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:02.841219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dc698-beed-4df2-be30-7ff276a5b13d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:02.855457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 304/604] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:02.862332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5097c-ebd6-49de-a6f6-dce3c84b2e20 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:02.945409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:02.989338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5097c-ebd6-49de-a6f6-dce3c84b2e20 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:03.027126 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55199371-58db-440f-9b7c-ca882d47e71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:03.027796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55199371-58db-440f-9b7c-ca882d47e71c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:03.028076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55199371-58db-440f-9b7c-ca882d47e71c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:03.028347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55199371-58db-440f-9b7c-ca882d47e71c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:03.035772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55199371-58db-440f-9b7c-ca882d47e71c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:03.037236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55199371-58db-440f-9b7c-ca882d47e71c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-55199371-58db-440f-9b7c-ca882d47e71c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:03.050089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:03.051079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:03.052032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:03.052032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:03.052032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:03.052032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:03.052032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:03.052032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:03.052032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:03.052032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:03.052032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 301/605] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:03.056112 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dc698-beed-4df2-be30-7ff276a5b13d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:03.088917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:03.167381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dc698-beed-4df2-be30-7ff276a5b13d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:03.204290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536129b4-59a0-4de1-aa07-aa3eb9fe2b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:03.204934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536129b4-59a0-4de1-aa07-aa3eb9fe2b4f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:03.205248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536129b4-59a0-4de1-aa07-aa3eb9fe2b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:03.205534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536129b4-59a0-4de1-aa07-aa3eb9fe2b4f None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:03.206228 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536129b4-59a0-4de1-aa07-aa3eb9fe2b4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:03.206527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536129b4-59a0-4de1-aa07-aa3eb9fe2b4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:03.207983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536129b4-59a0-4de1-aa07-aa3eb9fe2b4f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:03.222668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 305/606] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:03.228623 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55199371-58db-440f-9b7c-ca882d47e71c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:03.346974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317315ea-1535-420d-acbe-8f489f5d92b8 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:03.347265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317315ea-1535-420d-acbe-8f489f5d92b8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:03.347428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317315ea-1535-420d-acbe-8f489f5d92b8 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:03.349048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 306/607] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:10:03.353850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac15453-38a0-4faa-a160-c2b13dde0ba3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:03.354133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac15453-38a0-4faa-a160-c2b13dde0ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:03.354404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac15453-38a0-4faa-a160-c2b13dde0ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:03.361723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55199371-58db-440f-9b7c-ca882d47e71c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:03.374795 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1ac15453-38a0-4faa-a160-c2b13dde0ba3 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:03.376309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1ac15453-38a0-4faa-a160-c2b13dde0ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:03.413167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:03.429380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9267fb-71c6-409e-8218-850b97a953f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:03.430051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9267fb-71c6-409e-8218-850b97a953f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:03.430328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9267fb-71c6-409e-8218-850b97a953f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:03.430598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9267fb-71c6-409e-8218-850b97a953f4 None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/9a89a2e6aeb4495a84a61bd82b02bfc4/roles/dd50550e774349d6af84f593eeeec10d` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:03.450922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 307/608] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:03.466399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9267fb-71c6-409e-8218-850b97a953f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=9a89a2e6aeb4495a84a61bd82b02bfc4, role_id=dd50550e774349d6af84f593eeeec10d)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:03.468553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9267fb-71c6-409e-8218-850b97a953f4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:03.551327 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 302/609] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:10:03 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/9a89a2e6aeb4495a84a61bd82b02bfc4/roles/dd50550e774349d6af84f593eeeec10d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:10:03.657343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b5971-aace-49f4-8132-f9449b6991c0 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:03.657962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b5971-aace-49f4-8132-f9449b6991c0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:03.658536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b5971-aace-49f4-8132-f9449b6991c0 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:03.660457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 308/610] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:03.665631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f658bb5-73e6-4d4c-8417-e7c03a3f7fb3 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:03.666102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f658bb5-73e6-4d4c-8417-e7c03a3f7fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:03.666179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f658bb5-73e6-4d4c-8417-e7c03a3f7fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:03.690385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-9f658bb5-73e6-4d4c-8417-e7c03a3f7fb3 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:03.691675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9f658bb5-73e6-4d4c-8417-e7c03a3f7fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:03.772597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b5971-aace-49f4-8132-f9449b6991c0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:03.779377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 303/611] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:03.799392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:03.903223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b5971-aace-49f4-8132-f9449b6991c0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:03.938203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd1ec3-b7e8-4021-94e0-9e19a2ac40d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:03.938761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd1ec3-b7e8-4021-94e0-9e19a2ac40d4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:03.939037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd1ec3-b7e8-4021-94e0-9e19a2ac40d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:03.939308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd1ec3-b7e8-4021-94e0-9e19a2ac40d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:03.949683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd1ec3-b7e8-4021-94e0-9e19a2ac40d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-83bd1ec3-b7e8-4021-94e0-9e19a2ac40d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:03.955148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:03.956091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:03.956920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 309/612] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:03.964233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f658bb5-73e6-4d4c-8417-e7c03a3f7fb3 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:04.091050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f658bb5-73e6-4d4c-8417-e7c03a3f7fb3 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:04.098758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:04.126402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a0f8f-0a12-4ccf-bfe3-b73d7de5817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:04.127027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a0f8f-0a12-4ccf-bfe3-b73d7de5817c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:04.127302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a0f8f-0a12-4ccf-bfe3-b73d7de5817c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:04.127596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a0f8f-0a12-4ccf-bfe3-b73d7de5817c None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:04.128271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a0f8f-0a12-4ccf-bfe3-b73d7de5817c None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:04.128635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a0f8f-0a12-4ccf-bfe3-b73d7de5817c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:04.130046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a0f8f-0a12-4ccf-bfe3-b73d7de5817c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:04.139964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 304/613] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:10:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:04.145500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd1ec3-b7e8-4021-94e0-9e19a2ac40d4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:04.151167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a0f8f-0a12-4ccf-bfe3-b73d7de5817c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:04.288455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:04.316146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a0f8f-0a12-4ccf-bfe3-b73d7de5817c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:04.331031 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd1ec3-b7e8-4021-94e0-9e19a2ac40d4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:04.381213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdc4ef-5551-4ff8-9052-d4d513fc9a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:04.381852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdc4ef-5551-4ff8-9052-d4d513fc9a86 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:04.382133 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdc4ef-5551-4ff8-9052-d4d513fc9a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:04.382404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdc4ef-5551-4ff8-9052-d4d513fc9a86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:04.393072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53083d72-333e-49b7-b6e2-13eecadec629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:04.393698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53083d72-333e-49b7-b6e2-13eecadec629 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:04.393971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53083d72-333e-49b7-b6e2-13eecadec629 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:04.394239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53083d72-333e-49b7-b6e2-13eecadec629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:04.405400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdc4ef-5551-4ff8-9052-d4d513fc9a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-78fdc4ef-5551-4ff8-9052-d4d513fc9a86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:04.412466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:04.417169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:04.418005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 305/614] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:04.420301 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53083d72-333e-49b7-b6e2-13eecadec629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:04.422506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53083d72-333e-49b7-b6e2-13eecadec629 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:04.423754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdc4ef-5551-4ff8-9052-d4d513fc9a86 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-53083d72-333e-49b7-b6e2-13eecadec629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:04.446558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:04.451967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:04.452935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:04.452935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:04.452935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:04.455119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 310/615] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:04.460447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:04.462083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53083d72-333e-49b7-b6e2-13eecadec629 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:04.577849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdc4ef-5551-4ff8-9052-d4d513fc9a86 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:04.640657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53083d72-333e-49b7-b6e2-13eecadec629 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:04.642131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327f54b-0bc6-454a-9cef-f09bc25c901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:04.643741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327f54b-0bc6-454a-9cef-f09bc25c901c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:04.644239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327f54b-0bc6-454a-9cef-f09bc25c901c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:04.645311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327f54b-0bc6-454a-9cef-f09bc25c901c None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:04.646164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327f54b-0bc6-454a-9cef-f09bc25c901c None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:04.646841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327f54b-0bc6-454a-9cef-f09bc25c901c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:04.649220 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327f54b-0bc6-454a-9cef-f09bc25c901c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:04.665831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 306/616] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:10:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:04.673747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327f54b-0bc6-454a-9cef-f09bc25c901c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:04.699494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:04.714654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24798864-a161-449a-adc8-d88088c82ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:04.715591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24798864-a161-449a-adc8-d88088c82ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:04.716028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24798864-a161-449a-adc8-d88088c82ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:04.731656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24798864-a161-449a-adc8-d88088c82ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:04.731656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24798864-a161-449a-adc8-d88088c82ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:04.731656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24798864-a161-449a-adc8-d88088c82ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:04.731656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24798864-a161-449a-adc8-d88088c82ba6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:04.737127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 311/617] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:04.742300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24798864-a161-449a-adc8-d88088c82ba6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:04.836016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:04.854302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327f54b-0bc6-454a-9cef-f09bc25c901c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:04.893444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c60712-0288-4803-a3c5-044d7c298f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:04.894196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c60712-0288-4803-a3c5-044d7c298f16 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:04.894481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c60712-0288-4803-a3c5-044d7c298f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:04.894757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c60712-0288-4803-a3c5-044d7c298f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:04.913059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24798864-a161-449a-adc8-d88088c82ba6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:04.914248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c60712-0288-4803-a3c5-044d7c298f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:04.915989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c60712-0288-4803-a3c5-044d7c298f16 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-39c60712-0288-4803-a3c5-044d7c298f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:04.942855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:04.943874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:04.944931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:04.944931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:04.944931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:04.948999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479aa0a-77a3-46bd-a361-a0d366f8eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:04.949716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479aa0a-77a3-46bd-a361-a0d366f8eddc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:04.950064 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479aa0a-77a3-46bd-a361-a0d366f8eddc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:04.950404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479aa0a-77a3-46bd-a361-a0d366f8eddc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:04.953767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 307/618] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:04.958843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479aa0a-77a3-46bd-a361-a0d366f8eddc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:04.959065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c60712-0288-4803-a3c5-044d7c298f16 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:04.960434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479aa0a-77a3-46bd-a361-a0d366f8eddc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7479aa0a-77a3-46bd-a361-a0d366f8eddc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:04.973417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:04.975609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:04.976617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 312/619] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:04.984836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479aa0a-77a3-46bd-a361-a0d366f8eddc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:05.075307 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c60712-0288-4803-a3c5-044d7c298f16 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:05.082406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:05.112817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a29135-adba-40e0-be0f-416a57ac7a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:05.113347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29135-adba-40e0-be0f-416a57ac7a59 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.113629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29135-adba-40e0-be0f-416a57ac7a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.113944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29135-adba-40e0-be0f-416a57ac7a59 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.114595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29135-adba-40e0-be0f-416a57ac7a59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:05.114968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29135-adba-40e0-be0f-416a57ac7a59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:05.116406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29135-adba-40e0-be0f-416a57ac7a59 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:05.130990 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 308/620] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:05.132456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6f8e7-28bd-4b2e-b65a-46ee9a3a1d50 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.132745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6f8e7-28bd-4b2e-b65a-46ee9a3a1d50 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.133022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6f8e7-28bd-4b2e-b65a-46ee9a3a1d50 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.134564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 309/621] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:05.136383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6f8e7-28bd-4b2e-b65a-46ee9a3a1d50 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:05.142339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479aa0a-77a3-46bd-a361-a0d366f8eddc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:05.148344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:05.179687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b41e51-80a9-4198-9d84-430bcd5bc351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:05.180324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b41e51-80a9-4198-9d84-430bcd5bc351 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.180560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b41e51-80a9-4198-9d84-430bcd5bc351 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.180830 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b41e51-80a9-4198-9d84-430bcd5bc351 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.181552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b41e51-80a9-4198-9d84-430bcd5bc351 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:05.181858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b41e51-80a9-4198-9d84-430bcd5bc351 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:05.183372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b41e51-80a9-4198-9d84-430bcd5bc351 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:05.203111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 313/622] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Sep 5 12:10:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:05.204524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb963a-d62a-48dd-8623-55506dfdf72b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.204806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb963a-d62a-48dd-8623-55506dfdf72b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.205079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb963a-d62a-48dd-8623-55506dfdf72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.224899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4fcb963a-d62a-48dd-8623-55506dfdf72b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:05.226082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4fcb963a-d62a-48dd-8623-55506dfdf72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:05.249360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6f8e7-28bd-4b2e-b65a-46ee9a3a1d50 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:05.287059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63069c-333d-4c58-a3b8-6c0b706e5842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:05.287726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63069c-333d-4c58-a3b8-6c0b706e5842 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.288009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63069c-333d-4c58-a3b8-6c0b706e5842 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.288285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63069c-333d-4c58-a3b8-6c0b706e5842 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.296364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63069c-333d-4c58-a3b8-6c0b706e5842 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:05.297535 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 314/623] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:05.297962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63069c-333d-4c58-a3b8-6c0b706e5842 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:05.298559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb963a-d62a-48dd-8623-55506dfdf72b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-8c63069c-333d-4c58-a3b8-6c0b706e5842 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:05.311701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:05.313884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:05.316125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 310/624] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Sep 5 12:10:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:05.323668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63069c-333d-4c58-a3b8-6c0b706e5842 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:05.341645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:05.396378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:05.411903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb963a-d62a-48dd-8623-55506dfdf72b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:05.440300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63069c-333d-4c58-a3b8-6c0b706e5842 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:05.449135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e2621-daa3-4e72-80db-86f12823bfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:05.449766 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e2621-daa3-4e72-80db-86f12823bfc5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.450041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e2621-daa3-4e72-80db-86f12823bfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.450307 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e2621-daa3-4e72-80db-86f12823bfc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.462094 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e2621-daa3-4e72-80db-86f12823bfc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:05.463890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e2621-daa3-4e72-80db-86f12823bfc5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-4d5e2621-daa3-4e72-80db-86f12823bfc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:05.479808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:05.482145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:05.484011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:05.484011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:05.484011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:05.484011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 315/625] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:05.487160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ab794-7cc3-4d39-a336-b7210776be42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:05.487644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e2621-daa3-4e72-80db-86f12823bfc5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:05.487944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ab794-7cc3-4d39-a336-b7210776be42 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.488293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ab794-7cc3-4d39-a336-b7210776be42 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.488667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ab794-7cc3-4d39-a336-b7210776be42 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.489485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ab794-7cc3-4d39-a336-b7210776be42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:05.489906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ab794-7cc3-4d39-a336-b7210776be42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:05.491997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ab794-7cc3-4d39-a336-b7210776be42 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:05.512224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 311/626] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Sep 5 12:10:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:05.513463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f6ed-b1ae-40fb-a34d-44e64329fd90 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.513851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f6ed-b1ae-40fb-a34d-44e64329fd90 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.514206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f6ed-b1ae-40fb-a34d-44e64329fd90 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.515920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 312/627] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:05.518894 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810f6ed-b1ae-40fb-a34d-44e64329fd90 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:05.586500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:05.599414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e2621-daa3-4e72-80db-86f12823bfc5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:05.630876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810f6ed-b1ae-40fb-a34d-44e64329fd90 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:05.636802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f8720-886a-4a0d-a490-0758073d127d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:05.637406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8720-886a-4a0d-a490-0758073d127d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.637680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8720-886a-4a0d-a490-0758073d127d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.637946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8720-886a-4a0d-a490-0758073d127d None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.638584 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f8720-886a-4a0d-a490-0758073d127d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:05.638875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f8720-886a-4a0d-a490-0758073d127d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:05.640404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f8720-886a-4a0d-a490-0758073d127d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:05.653054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:05.658271 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 316/628] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:05.659636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fca27-48af-4fc2-8003-7ab1b69cf186 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.659918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fca27-48af-4fc2-8003-7ab1b69cf186 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.660186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fca27-48af-4fc2-8003-7ab1b69cf186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.668079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7140a-e599-4793-bb6a-0a4edcca78b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:05.668721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7140a-e599-4793-bb6a-0a4edcca78b5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.668939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7140a-e599-4793-bb6a-0a4edcca78b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.669205 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7140a-e599-4793-bb6a-0a4edcca78b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.678529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-605fca27-48af-4fc2-8003-7ab1b69cf186 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:05.679742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-605fca27-48af-4fc2-8003-7ab1b69cf186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:05.741698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7140a-e599-4793-bb6a-0a4edcca78b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:05.743151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7140a-e599-4793-bb6a-0a4edcca78b5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:05.744936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 317/629] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:05.745513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fca27-48af-4fc2-8003-7ab1b69cf186 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:05.832114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:05.854752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fca27-48af-4fc2-8003-7ab1b69cf186 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:05.882789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 313/630] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:10:05.887576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7140a-e599-4793-bb6a-0a4edcca78b5 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:05.897071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66986d5-d8a6-4061-b872-d77ed263f494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:05.897992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66986d5-d8a6-4061-b872-d77ed263f494 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:05.898438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66986d5-d8a6-4061-b872-d77ed263f494 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:05.898879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66986d5-d8a6-4061-b872-d77ed263f494 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:05.909303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:05.912576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66986d5-d8a6-4061-b872-d77ed263f494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e66986d5-d8a6-4061-b872-d77ed263f494 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:05.917725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:05.924752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 318/631] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:05.925595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66986d5-d8a6-4061-b872-d77ed263f494 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:05.997637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7140a-e599-4793-bb6a-0a4edcca78b5 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.032624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a673a9-ae04-46ea-9914-33af3c1b0a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.032866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66986d5-d8a6-4061-b872-d77ed263f494 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.033331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a673a9-ae04-46ea-9914-33af3c1b0a10 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.033556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a673a9-ae04-46ea-9914-33af3c1b0a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.033837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a673a9-ae04-46ea-9914-33af3c1b0a10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.040353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a673a9-ae04-46ea-9914-33af3c1b0a10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:06.041801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a673a9-ae04-46ea-9914-33af3c1b0a10 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-47a673a9-ae04-46ea-9914-33af3c1b0a10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:06.053524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:06.054438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:06.055325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:06.055325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:06.055325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:06.055325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:06.055325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:06.055325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:06.055325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:06.055325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:06.055325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 314/632] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:06.055849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a673a9-ae04-46ea-9914-33af3c1b0a10 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.071265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb98ed-45d0-4813-bbc9-428c654e823d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.071965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb98ed-45d0-4813-bbc9-428c654e823d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.072242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb98ed-45d0-4813-bbc9-428c654e823d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.072511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb98ed-45d0-4813-bbc9-428c654e823d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.080223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:06.084232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb98ed-45d0-4813-bbc9-428c654e823d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:06.085746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb98ed-45d0-4813-bbc9-428c654e823d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-57cb98ed-45d0-4813-bbc9-428c654e823d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:06.100683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:06.101822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:06.103089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:06.103089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:06.103089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:06.103089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 319/633] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:06.103089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c88169-c35c-4ed3-a3dc-572fff6b2771 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.103367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c88169-c35c-4ed3-a3dc-572fff6b2771 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.103603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c88169-c35c-4ed3-a3dc-572fff6b2771 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.106120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 320/634] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:06.106120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c88169-c35c-4ed3-a3dc-572fff6b2771 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.141433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:06.161095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a673a9-ae04-46ea-9914-33af3c1b0a10 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.195647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11731ae2-9a9d-498c-97b6-f3172db3052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.196251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11731ae2-9a9d-498c-97b6-f3172db3052a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.196535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11731ae2-9a9d-498c-97b6-f3172db3052a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.196807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11731ae2-9a9d-498c-97b6-f3172db3052a None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.197463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11731ae2-9a9d-498c-97b6-f3172db3052a None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:06.197762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11731ae2-9a9d-498c-97b6-f3172db3052a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:06.199134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11731ae2-9a9d-498c-97b6-f3172db3052a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:06.207737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c88169-c35c-4ed3-a3dc-572fff6b2771 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.208192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 315/635] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:10:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:06.208734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11731ae2-9a9d-498c-97b6-f3172db3052a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.245863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc8f8f-0a41-41fc-8ada-099193e13a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.246486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc8f8f-0a41-41fc-8ada-099193e13a8a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.246763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc8f8f-0a41-41fc-8ada-099193e13a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.247032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc8f8f-0a41-41fc-8ada-099193e13a8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.256959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc8f8f-0a41-41fc-8ada-099193e13a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-73bc8f8f-0a41-41fc-8ada-099193e13a8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:06.262130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:06.267775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 321/636] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:06.268851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc8f8f-0a41-41fc-8ada-099193e13a8a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.314381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11731ae2-9a9d-498c-97b6-f3172db3052a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.318154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:06.348799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e5872-6831-4af9-b88f-8a1f1bf4608f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.349396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e5872-6831-4af9-b88f-8a1f1bf4608f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.349674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e5872-6831-4af9-b88f-8a1f1bf4608f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.349941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e5872-6831-4af9-b88f-8a1f1bf4608f None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.350616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e5872-6831-4af9-b88f-8a1f1bf4608f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:06.350913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e5872-6831-4af9-b88f-8a1f1bf4608f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:06.352401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e5872-6831-4af9-b88f-8a1f1bf4608f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:06.364853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 316/637] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:06.366049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6cdd1-4914-434b-95a8-40b9c78b8274 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.366331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6cdd1-4914-434b-95a8-40b9c78b8274 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.366597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6cdd1-4914-434b-95a8-40b9c78b8274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.372663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:06.373376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc8f8f-0a41-41fc-8ada-099193e13a8a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.384196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ebb6cdd1-4914-434b-95a8-40b9c78b8274 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:06.385426 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ebb6cdd1-4914-434b-95a8-40b9c78b8274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:06.407147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4423921-2470-48f4-a4d7-65b6998929b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.407768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4423921-2470-48f4-a4d7-65b6998929b1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.408046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4423921-2470-48f4-a4d7-65b6998929b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.408319 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4423921-2470-48f4-a4d7-65b6998929b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.408952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4423921-2470-48f4-a4d7-65b6998929b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:06.409251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4423921-2470-48f4-a4d7-65b6998929b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:06.410593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4423921-2470-48f4-a4d7-65b6998929b1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:06.422145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 322/638] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:06.422882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4423921-2470-48f4-a4d7-65b6998929b1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.454181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 317/639] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:06.455005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6cdd1-4914-434b-95a8-40b9c78b8274 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.527471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4423921-2470-48f4-a4d7-65b6998929b1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.538579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:06.562651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a5e66-2547-419d-892f-0215d179b8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.563281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a5e66-2547-419d-892f-0215d179b8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.563576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a5e66-2547-419d-892f-0215d179b8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.563847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a5e66-2547-419d-892f-0215d179b8e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.570768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6cdd1-4914-434b-95a8-40b9c78b8274 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.574776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a5e66-2547-419d-892f-0215d179b8e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:06.576358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a5e66-2547-419d-892f-0215d179b8e8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-768a5e66-2547-419d-892f-0215d179b8e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:06.593970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:06.594938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:06.595872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:06.595872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:06.595872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:06.595872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 323/640] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:06.595872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a5e66-2547-419d-892f-0215d179b8e8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.608776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:06.613914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42ee23-a71f-447d-8ade-619c7f1c4e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.614520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42ee23-a71f-447d-8ade-619c7f1c4e32 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.614798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42ee23-a71f-447d-8ade-619c7f1c4e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.615068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42ee23-a71f-447d-8ade-619c7f1c4e32 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.615748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42ee23-a71f-447d-8ade-619c7f1c4e32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:06.616066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42ee23-a71f-447d-8ade-619c7f1c4e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:06.617399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42ee23-a71f-447d-8ade-619c7f1c4e32 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:06.630003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 318/641] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:10:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:06.630003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42ee23-a71f-447d-8ade-619c7f1c4e32 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.708026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a5e66-2547-419d-892f-0215d179b8e8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.732324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42ee23-a71f-447d-8ade-619c7f1c4e32 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.741522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3e543-e215-47af-baee-6b66892e8b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.742173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3e543-e215-47af-baee-6b66892e8b59 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.742449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3e543-e215-47af-baee-6b66892e8b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.742720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3e543-e215-47af-baee-6b66892e8b59 None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/b48aa41ff66b42329926bc42379d9800/roles/dd50550e774349d6af84f593eeeec10d` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.757340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3e543-e215-47af-baee-6b66892e8b59 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=b48aa41ff66b42329926bc42379d9800, role_id=dd50550e774349d6af84f593eeeec10d)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:06.758749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3e543-e215-47af-baee-6b66892e8b59 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:06.766663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89e4d-8914-4a32-89a4-6f615d45c6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.767278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89e4d-8914-4a32-89a4-6f615d45c6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.767579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89e4d-8914-4a32-89a4-6f615d45c6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.767856 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89e4d-8914-4a32-89a4-6f615d45c6e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.774286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89e4d-8914-4a32-89a4-6f615d45c6e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:06.775734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89e4d-8914-4a32-89a4-6f615d45c6e7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-9bf89e4d-8914-4a32-89a4-6f615d45c6e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:06.787009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:06.788116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:06.789026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:06.789026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:06.789026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:06.789026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:06.789026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:06.789026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:06.789026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:06.789026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:06.789026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 319/642] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:06.789026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89e4d-8914-4a32-89a4-6f615d45c6e7 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.843174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:06.847148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 324/643] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:10:06 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/b48aa41ff66b42329926bc42379d9800/roles/dd50550e774349d6af84f593eeeec10d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:10:06.847400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3e543-e215-47af-baee-6b66892e8b59 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.863245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:06.900914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89e4d-8914-4a32-89a4-6f615d45c6e7 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:06.939236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd551fb-0062-498d-9f0e-84f6c3e4ee68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:06.939913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd551fb-0062-498d-9f0e-84f6c3e4ee68 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:06.940191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd551fb-0062-498d-9f0e-84f6c3e4ee68 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:06.940815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd551fb-0062-498d-9f0e-84f6c3e4ee68 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:06.941176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd551fb-0062-498d-9f0e-84f6c3e4ee68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:06.941477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd551fb-0062-498d-9f0e-84f6c3e4ee68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:06.943091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd551fb-0062-498d-9f0e-84f6c3e4ee68 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:06.961305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 320/644] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:06.961982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd551fb-0062-498d-9f0e-84f6c3e4ee68 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:06.962410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3e543-e215-47af-baee-6b66892e8b59 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.010602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f402dbf-86fe-4e34-a4ac-fafdb2264c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.011225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f402dbf-86fe-4e34-a4ac-fafdb2264c6a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.011499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f402dbf-86fe-4e34-a4ac-fafdb2264c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.011796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f402dbf-86fe-4e34-a4ac-fafdb2264c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:07.022081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f402dbf-86fe-4e34-a4ac-fafdb2264c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:07.023512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f402dbf-86fe-4e34-a4ac-fafdb2264c6a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7f402dbf-86fe-4e34-a4ac-fafdb2264c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:07.038060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:07.039216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:07.040444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:07.040444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:07.040444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:07.040444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 325/645] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:07.040444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f402dbf-86fe-4e34-a4ac-fafdb2264c6a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:07.090996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd551fb-0062-498d-9f0e-84f6c3e4ee68 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.106237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:07.114477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:07.127500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b10a8-2da7-40ff-9a68-b8068063d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.128225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b10a8-2da7-40ff-9a68-b8068063d241 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.128560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b10a8-2da7-40ff-9a68-b8068063d241 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.128832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b10a8-2da7-40ff-9a68-b8068063d241 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:07.141818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b10a8-2da7-40ff-9a68-b8068063d241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-580b10a8-2da7-40ff-9a68-b8068063d241 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:07.148836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:07.150091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:07.150928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:07.158450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f402dbf-86fe-4e34-a4ac-fafdb2264c6a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.158570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 321/646] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:07.158570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b10a8-2da7-40ff-9a68-b8068063d241 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:07.190463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020c82d-fd29-4011-a793-21bd21b3f728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.191161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020c82d-fd29-4011-a793-21bd21b3f728 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.191598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020c82d-fd29-4011-a793-21bd21b3f728 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.191939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020c82d-fd29-4011-a793-21bd21b3f728 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:07.192713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020c82d-fd29-4011-a793-21bd21b3f728 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:07.193093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020c82d-fd29-4011-a793-21bd21b3f728 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:07.194474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020c82d-fd29-4011-a793-21bd21b3f728 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:07.206887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 326/647] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:07.212283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020c82d-fd29-4011-a793-21bd21b3f728 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:07.273532 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b10a8-2da7-40ff-9a68-b8068063d241 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.307953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a622e5-b069-4cc2-b486-8403baecf7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.308574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a622e5-b069-4cc2-b486-8403baecf7fa None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.308847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a622e5-b069-4cc2-b486-8403baecf7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.309121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a622e5-b069-4cc2-b486-8403baecf7fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:07.317493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a622e5-b069-4cc2-b486-8403baecf7fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:07.318700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020c82d-fd29-4011-a793-21bd21b3f728 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.318930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a622e5-b069-4cc2-b486-8403baecf7fa None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-51a622e5-b069-4cc2-b486-8403baecf7fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:07.330101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:07.331062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 322/648] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:07.332090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a622e5-b069-4cc2-b486-8403baecf7fa None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:07.350556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:07.362053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445744bd-539b-4dc3-8407-2db9b1b6dd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.364349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445744bd-539b-4dc3-8407-2db9b1b6dd62 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.364349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445744bd-539b-4dc3-8407-2db9b1b6dd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.364349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445744bd-539b-4dc3-8407-2db9b1b6dd62 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:07.364349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445744bd-539b-4dc3-8407-2db9b1b6dd62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:07.364349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445744bd-539b-4dc3-8407-2db9b1b6dd62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:07.365532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445744bd-539b-4dc3-8407-2db9b1b6dd62 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:07.370412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:07.377151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 327/649] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:07.382098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445744bd-539b-4dc3-8407-2db9b1b6dd62 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:07.442239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a622e5-b069-4cc2-b486-8403baecf7fa None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.479218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292071e-3477-4d51-82ea-8ccfa2b00651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.479812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292071e-3477-4d51-82ea-8ccfa2b00651 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.480099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292071e-3477-4d51-82ea-8ccfa2b00651 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.480373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292071e-3477-4d51-82ea-8ccfa2b00651 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:07.481026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292071e-3477-4d51-82ea-8ccfa2b00651 None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:07.481332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292071e-3477-4d51-82ea-8ccfa2b00651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:07.482709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292071e-3477-4d51-82ea-8ccfa2b00651 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:07.489746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445744bd-539b-4dc3-8407-2db9b1b6dd62 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.495590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 323/650] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:10:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:07.495590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292071e-3477-4d51-82ea-8ccfa2b00651 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:07.529057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d7725-296a-4346-89e2-5e56263d563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.529653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d7725-296a-4346-89e2-5e56263d563f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.530450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d7725-296a-4346-89e2-5e56263d563f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.530716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d7725-296a-4346-89e2-5e56263d563f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:07.531431 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d7725-296a-4346-89e2-5e56263d563f None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=7956a7748ec94a618eb466f5536220b0) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:07.531757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d7725-296a-4346-89e2-5e56263d563f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:07.533282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d7725-296a-4346-89e2-5e56263d563f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:07.539863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 328/651] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:07 2022] GET /identity/v3/role_assignments?user.id=7956a7748ec94a618eb466f5536220b0&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:07.540665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d7725-296a-4346-89e2-5e56263d563f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:07.604815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:07.607458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:07.614049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292071e-3477-4d51-82ea-8ccfa2b00651 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.660626 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d41a2-cc88-4590-b428-f3bc6cb8c0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.661238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d41a2-cc88-4590-b428-f3bc6cb8c0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.661513 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d41a2-cc88-4590-b428-f3bc6cb8c0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.661781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d41a2-cc88-4590-b428-f3bc6cb8c0a3 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:07.662574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d41a2-cc88-4590-b428-f3bc6cb8c0a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:07.662883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d41a2-cc88-4590-b428-f3bc6cb8c0a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:07.664364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d41a2-cc88-4590-b428-f3bc6cb8c0a3 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:07.669269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d7725-296a-4346-89e2-5e56263d563f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.686110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 324/652] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Sep 5 12:10:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:07.687029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d41a2-cc88-4590-b428-f3bc6cb8c0a3 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:07.725247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae00ef8-2dca-42fc-a74d-ce1bda6a8921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.725883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae00ef8-2dca-42fc-a74d-ce1bda6a8921 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.726149 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae00ef8-2dca-42fc-a74d-ce1bda6a8921 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.726475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae00ef8-2dca-42fc-a74d-ce1bda6a8921 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:07.737499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae00ef8-2dca-42fc-a74d-ce1bda6a8921 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:07.739206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae00ef8-2dca-42fc-a74d-ce1bda6a8921 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-2ae00ef8-2dca-42fc-a74d-ce1bda6a8921 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:07.754036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:07.755007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:07.756001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 329/653] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:07.756687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae00ef8-2dca-42fc-a74d-ce1bda6a8921 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:07.815243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d41a2-cc88-4590-b428-f3bc6cb8c0a3 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.857578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a39df-c3e8-4ea6-8545-448af97f611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.858620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a39df-c3e8-4ea6-8545-448af97f611c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.859129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a39df-c3e8-4ea6-8545-448af97f611c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.859630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a39df-c3e8-4ea6-8545-448af97f611c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:07.870891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a39df-c3e8-4ea6-8545-448af97f611c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:07.872851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a39df-c3e8-4ea6-8545-448af97f611c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:07.882775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae00ef8-2dca-42fc-a74d-ce1bda6a8921 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-cd9a39df-c3e8-4ea6-8545-448af97f611c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:07.889078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:07.890543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:07.891467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:07.891467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:07.891467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:07.892963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:07.894516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 325/654] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:07.895606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a39df-c3e8-4ea6-8545-448af97f611c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:07.905772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:07.924496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f631b0e-594e-4212-a828-9ffa774927c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:07.925152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f631b0e-594e-4212-a828-9ffa774927c0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:07.925586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f631b0e-594e-4212-a828-9ffa774927c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:07.925872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f631b0e-594e-4212-a828-9ffa774927c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:08.011285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a39df-c3e8-4ea6-8545-448af97f611c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:08.013995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f631b0e-594e-4212-a828-9ffa774927c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:08.016765 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f631b0e-594e-4212-a828-9ffa774927c0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:08.049915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682338e-125b-457b-862b-7c686de3016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:08.050996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682338e-125b-457b-862b-7c686de3016b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:08.051506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682338e-125b-457b-862b-7c686de3016b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:08.052108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682338e-125b-457b-862b-7c686de3016b None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:08.053233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682338e-125b-457b-862b-7c686de3016b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:08.053741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682338e-125b-457b-862b-7c686de3016b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:08.055433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682338e-125b-457b-862b-7c686de3016b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:08.075787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 326/655] 10.222.0.62 () {56 vars in 1137 bytes} [Mon Sep 5 12:10:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:08.077424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682338e-125b-457b-862b-7c686de3016b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:08.145309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 330/656] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:10:08.146594 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f631b0e-594e-4212-a828-9ffa774927c0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:08.157825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:08.161914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:08.211384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682338e-125b-457b-862b-7c686de3016b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:08.257423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97519f-3b1a-4f5d-9e59-bbeffb39a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:08.258425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97519f-3b1a-4f5d-9e59-bbeffb39a119 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:08.258929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97519f-3b1a-4f5d-9e59-bbeffb39a119 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:08.259439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97519f-3b1a-4f5d-9e59-bbeffb39a119 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:08.260306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97519f-3b1a-4f5d-9e59-bbeffb39a119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:08.260958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97519f-3b1a-4f5d-9e59-bbeffb39a119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:08.262577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97519f-3b1a-4f5d-9e59-bbeffb39a119 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:08.267495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f631b0e-594e-4212-a828-9ffa774927c0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:08.276246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 327/657] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:08.277392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97519f-3b1a-4f5d-9e59-bbeffb39a119 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:08.316396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5045288-5437-43c1-ac1f-98a819c89608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:08.316884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5045288-5437-43c1-ac1f-98a819c89608 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:08.317163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5045288-5437-43c1-ac1f-98a819c89608 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:08.317437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5045288-5437-43c1-ac1f-98a819c89608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:08.398952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97519f-3b1a-4f5d-9e59-bbeffb39a119 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:08.405112 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5045288-5437-43c1-ac1f-98a819c89608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:08.406627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5045288-5437-43c1-ac1f-98a819c89608 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:08.418150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:08.431225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:08.446214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b45633-6d15-40ef-8d6b-a06ee6db6096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:08.447151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b45633-6d15-40ef-8d6b-a06ee6db6096 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:08.447676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b45633-6d15-40ef-8d6b-a06ee6db6096 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:08.448153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b45633-6d15-40ef-8d6b-a06ee6db6096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:08.464122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b45633-6d15-40ef-8d6b-a06ee6db6096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:08.466620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b45633-6d15-40ef-8d6b-a06ee6db6096 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-e9b45633-6d15-40ef-8d6b-a06ee6db6096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:08.487176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:08.488168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:08.489054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:08.489054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:08.489054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:08.493404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 328/658] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:08.493994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b45633-6d15-40ef-8d6b-a06ee6db6096 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:08.530547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 331/659] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:10:08.530547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5045288-5437-43c1-ac1f-98a819c89608 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:08.605251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b45633-6d15-40ef-8d6b-a06ee6db6096 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:08.646849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b4697-6279-4497-aee8-8c0b96098386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:08.648568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b4697-6279-4497-aee8-8c0b96098386 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:08.649056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b4697-6279-4497-aee8-8c0b96098386 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:08.649751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b4697-6279-4497-aee8-8c0b96098386 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:08.658367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5045288-5437-43c1-ac1f-98a819c89608 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:08.659297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b4697-6279-4497-aee8-8c0b96098386 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:08.660964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b4697-6279-4497-aee8-8c0b96098386 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:08.668485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-ff2b4697-6279-4497-aee8-8c0b96098386 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:08.673107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:08.674091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:08.675079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 329/660] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:08.675717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b4697-6279-4497-aee8-8c0b96098386 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:08.702908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c489fb-474a-49b1-83f1-4b3ece5b47b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:08.705611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c489fb-474a-49b1-83f1-4b3ece5b47b1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:08.706046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c489fb-474a-49b1-83f1-4b3ece5b47b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:08.706751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c489fb-474a-49b1-83f1-4b3ece5b47b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:08.707991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c489fb-474a-49b1-83f1-4b3ece5b47b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:08.708630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c489fb-474a-49b1-83f1-4b3ece5b47b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:08.710627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c489fb-474a-49b1-83f1-4b3ece5b47b1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:08.715952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:08.728942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 332/661] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:08.734292 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c489fb-474a-49b1-83f1-4b3ece5b47b1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:08.793790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b4697-6279-4497-aee8-8c0b96098386 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:08.831488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31feede-1ff8-4a06-819d-05c1c3f724a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:08.832160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31feede-1ff8-4a06-819d-05c1c3f724a2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:08.832521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31feede-1ff8-4a06-819d-05c1c3f724a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:08.832796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31feede-1ff8-4a06-819d-05c1c3f724a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:08.844887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31feede-1ff8-4a06-819d-05c1c3f724a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:08.846480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31feede-1ff8-4a06-819d-05c1c3f724a2 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:08.848254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c489fb-474a-49b1-83f1-4b3ece5b47b1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-f31feede-1ff8-4a06-819d-05c1c3f724a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:08.861816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:08.863034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:08.864248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:08.864248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:08.864248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:08.867577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 330/662] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:08.868141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31feede-1ff8-4a06-819d-05c1c3f724a2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:08.884878 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a3fa8-f7e1-4ff9-b81a-cb8cde0a80ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:08.885582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3fa8-f7e1-4ff9-b81a-cb8cde0a80ea None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:08.885933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3fa8-f7e1-4ff9-b81a-cb8cde0a80ea None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:08.886270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3fa8-f7e1-4ff9-b81a-cb8cde0a80ea None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:08.886998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a3fa8-f7e1-4ff9-b81a-cb8cde0a80ea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:08.887299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a3fa8-f7e1-4ff9-b81a-cb8cde0a80ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:08.888789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a3fa8-f7e1-4ff9-b81a-cb8cde0a80ea None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:08.912749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 333/663] 10.222.0.62 () {56 vars in 1139 bytes} [Mon Sep 5 12:10:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:08.912749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c172d3-996f-49df-b07e-5e4441a7e2c8 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:08.912749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c172d3-996f-49df-b07e-5e4441a7e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:08.912749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c172d3-996f-49df-b07e-5e4441a7e2c8 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:08.912749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 334/664] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:08.912749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c172d3-996f-49df-b07e-5e4441a7e2c8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:08.939857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:08.968062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:08.980541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31feede-1ff8-4a06-819d-05c1c3f724a2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:09.015374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c172d3-996f-49df-b07e-5e4441a7e2c8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:09.016676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3914ae-4686-46a2-bdf2-650cfacfcad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:09.017396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3914ae-4686-46a2-bdf2-650cfacfcad9 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.017784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3914ae-4686-46a2-bdf2-650cfacfcad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.018064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3914ae-4686-46a2-bdf2-650cfacfcad9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.028355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3914ae-4686-46a2-bdf2-650cfacfcad9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:09.029799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3914ae-4686-46a2-bdf2-650cfacfcad9 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-fe3914ae-4686-46a2-bdf2-650cfacfcad9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:09.041394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:09.042459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:09.043505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:09.043505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:09.043505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:09.043505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:09.043505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:09.043505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:09.043505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:09.043505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:09.043505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 331/665] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:09.044086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3914ae-4686-46a2-bdf2-650cfacfcad9 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:09.059385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c640f83-7d44-472d-988f-07ca2095c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:09.060199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c640f83-7d44-472d-988f-07ca2095c8eb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.060635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c640f83-7d44-472d-988f-07ca2095c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.061045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c640f83-7d44-472d-988f-07ca2095c8eb None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.061836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c640f83-7d44-472d-988f-07ca2095c8eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:09.062275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c640f83-7d44-472d-988f-07ca2095c8eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:09.063918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c640f83-7d44-472d-988f-07ca2095c8eb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:09.083752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 335/666] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:09.083752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc56803-c71e-4362-a5ca-cb6e463930f1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.083752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc56803-c71e-4362-a5ca-cb6e463930f1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.083752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc56803-c71e-4362-a5ca-cb6e463930f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.102041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3cc56803-c71e-4362-a5ca-cb6e463930f1 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:09.103319 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3cc56803-c71e-4362-a5ca-cb6e463930f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:09.160331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3914ae-4686-46a2-bdf2-650cfacfcad9 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:09.173474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 336/667] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:09.176919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc56803-c71e-4362-a5ca-cb6e463930f1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:09.197373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18baeae-0486-459b-8870-6d6cb7abaf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:09.198136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18baeae-0486-459b-8870-6d6cb7abaf68 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.198571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18baeae-0486-459b-8870-6d6cb7abaf68 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.198941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18baeae-0486-459b-8870-6d6cb7abaf68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.206616 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:09.219446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:09.284967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18baeae-0486-459b-8870-6d6cb7abaf68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:09.286677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18baeae-0486-459b-8870-6d6cb7abaf68 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:09.318113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc56803-c71e-4362-a5ca-cb6e463930f1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:09.355673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69578c5e-634c-4a5e-a544-1bce53b2851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:09.356458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69578c5e-634c-4a5e-a544-1bce53b2851f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.356832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69578c5e-634c-4a5e-a544-1bce53b2851f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.357192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69578c5e-634c-4a5e-a544-1bce53b2851f None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.358008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69578c5e-634c-4a5e-a544-1bce53b2851f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:09.358417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69578c5e-634c-4a5e-a544-1bce53b2851f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:09.359983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69578c5e-634c-4a5e-a544-1bce53b2851f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:09.375940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 337/668] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:09.377065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69578c5e-634c-4a5e-a544-1bce53b2851f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:09.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 332/669] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:10:09.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18baeae-0486-459b-8870-6d6cb7abaf68 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:09.471043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:09.479113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:09.501826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69578c5e-634c-4a5e-a544-1bce53b2851f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:09.524526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18baeae-0486-459b-8870-6d6cb7abaf68 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:09.536152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad187d25-4f3e-450d-8494-0d2d752bb125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:09.536952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad187d25-4f3e-450d-8494-0d2d752bb125 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.537337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad187d25-4f3e-450d-8494-0d2d752bb125 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.537711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad187d25-4f3e-450d-8494-0d2d752bb125 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.545014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad187d25-4f3e-450d-8494-0d2d752bb125 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:09.546890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad187d25-4f3e-450d-8494-0d2d752bb125 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-ad187d25-4f3e-450d-8494-0d2d752bb125 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:09.558488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:09.559574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:09.565523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:09.565523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:09.565523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:09.565523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:09.565523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:09.565523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:09.565523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:09.565523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:09.565523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 338/670] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:09.565523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad187d25-4f3e-450d-8494-0d2d752bb125 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:09.571790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda5400-b527-4e23-8ace-1f48c9d00f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:09.572905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda5400-b527-4e23-8ace-1f48c9d00f36 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.573280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda5400-b527-4e23-8ace-1f48c9d00f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.574061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda5400-b527-4e23-8ace-1f48c9d00f36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.575209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda5400-b527-4e23-8ace-1f48c9d00f36 None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=237bbeebe2a949aba444353c1d2ee04f) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:09.575645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda5400-b527-4e23-8ace-1f48c9d00f36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:09.578516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda5400-b527-4e23-8ace-1f48c9d00f36 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:09.585312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 333/671] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:09 2022] GET /identity/v3/role_assignments?user.id=237bbeebe2a949aba444353c1d2ee04f&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:09.586112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda5400-b527-4e23-8ace-1f48c9d00f36 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:09.682829 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad187d25-4f3e-450d-8494-0d2d752bb125 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:09.715004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda5400-b527-4e23-8ace-1f48c9d00f36 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:09.721001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c55ce8-7d65-4045-8459-45600182519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:09.721644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c55ce8-7d65-4045-8459-45600182519c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.721919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c55ce8-7d65-4045-8459-45600182519c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.722186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c55ce8-7d65-4045-8459-45600182519c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.725187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:09.734358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:09.734987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c55ce8-7d65-4045-8459-45600182519c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:09.739786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c55ce8-7d65-4045-8459-45600182519c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-93c55ce8-7d65-4045-8459-45600182519c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:09.753750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:09.754754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:09.755762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:09.755762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:09.755762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:09.755762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 339/672] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:09.755762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c55ce8-7d65-4045-8459-45600182519c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:09.759415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7dcd9-e6ca-46d0-839a-85ec9cd03fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:09.760163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7dcd9-e6ca-46d0-839a-85ec9cd03fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.760447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7dcd9-e6ca-46d0-839a-85ec9cd03fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.760721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7dcd9-e6ca-46d0-839a-85ec9cd03fb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.772166 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7dcd9-e6ca-46d0-839a-85ec9cd03fb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-c2f7dcd9-e6ca-46d0-839a-85ec9cd03fb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:09.776964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:09.777881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:09.778724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 334/673] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:09.779432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7d8a-9d6c-418d-b6c3-468c46562721 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.785204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7d8a-9d6c-418d-b6c3-468c46562721 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.785204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7d8a-9d6c-418d-b6c3-468c46562721 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.787726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 335/674] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:09.788285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae7d8a-9d6c-418d-b6c3-468c46562721 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:09.872395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c55ce8-7d65-4045-8459-45600182519c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:09.902846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae7d8a-9d6c-418d-b6c3-468c46562721 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:09.917104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d8adb-e769-402d-96ea-02f5229f0108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:09.917771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d8adb-e769-402d-96ea-02f5229f0108 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.918056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d8adb-e769-402d-96ea-02f5229f0108 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.918327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d8adb-e769-402d-96ea-02f5229f0108 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.919029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d8adb-e769-402d-96ea-02f5229f0108 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:09.919334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d8adb-e769-402d-96ea-02f5229f0108 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:09.920819 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d8adb-e769-402d-96ea-02f5229f0108 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:09.938046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 340/675] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:09.938046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d8adb-e769-402d-96ea-02f5229f0108 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:09.945221 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cde10b-62d9-4b4c-bbf7-f0e580863120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:09.945832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cde10b-62d9-4b4c-bbf7-f0e580863120 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.946110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cde10b-62d9-4b4c-bbf7-f0e580863120 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.946495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cde10b-62d9-4b4c-bbf7-f0e580863120 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.947131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cde10b-62d9-4b4c-bbf7-f0e580863120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:09.947428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cde10b-62d9-4b4c-bbf7-f0e580863120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:09.948953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cde10b-62d9-4b4c-bbf7-f0e580863120 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:09.967107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 336/676] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:09.968442 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02844fec-3153-484b-baa6-5d0162a20ed6 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:09.968725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02844fec-3153-484b-baa6-5d0162a20ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:09.969000 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02844fec-3153-484b-baa6-5d0162a20ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:09.987365 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:09.987572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-02844fec-3153-484b-baa6-5d0162a20ed6 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:09.988946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-02844fec-3153-484b-baa6-5d0162a20ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:10.021410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:10.040665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d8adb-e769-402d-96ea-02f5229f0108 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:10.070971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 337/677] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:10.071689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02844fec-3153-484b-baa6-5d0162a20ed6 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:10.090286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7390b-9291-4f29-aebc-a2fcf869808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:10.091158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7390b-9291-4f29-aebc-a2fcf869808f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:10.091784 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7390b-9291-4f29-aebc-a2fcf869808f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:10.092424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7390b-9291-4f29-aebc-a2fcf869808f None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:10.093263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7390b-9291-4f29-aebc-a2fcf869808f None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:10.093874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7390b-9291-4f29-aebc-a2fcf869808f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:10.095447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7390b-9291-4f29-aebc-a2fcf869808f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:10.105231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 341/678] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:10:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:10.106374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7390b-9291-4f29-aebc-a2fcf869808f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:10.213955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02844fec-3153-484b-baa6-5d0162a20ed6 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:10.219892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7390b-9291-4f29-aebc-a2fcf869808f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:10.231269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:10.259972 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1f1b4-4e11-46e7-8b30-1df191f6ddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:10.260576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1f1b4-4e11-46e7-8b30-1df191f6ddcd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:10.260687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec6f23-c88a-4339-95f3-63844691d296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:10.260849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1f1b4-4e11-46e7-8b30-1df191f6ddcd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:10.261127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1f1b4-4e11-46e7-8b30-1df191f6ddcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:10.261964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec6f23-c88a-4339-95f3-63844691d296 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:10.262081 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1f1b4-4e11-46e7-8b30-1df191f6ddcd None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=8f33f1f3d1ca4436887c2346bd9fa149) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:10.262509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1f1b4-4e11-46e7-8b30-1df191f6ddcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:10.262800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec6f23-c88a-4339-95f3-63844691d296 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:10.263258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec6f23-c88a-4339-95f3-63844691d296 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:10.263983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1f1b4-4e11-46e7-8b30-1df191f6ddcd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:10.271386 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec6f23-c88a-4339-95f3-63844691d296 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:10.273182 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec6f23-c88a-4339-95f3-63844691d296 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:10.275734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 338/679] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:10 2022] GET /identity/v3/role_assignments?user.id=8f33f1f3d1ca4436887c2346bd9fa149&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:10.276903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1f1b4-4e11-46e7-8b30-1df191f6ddcd None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-74ec6f23-c88a-4339-95f3-63844691d296 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:10.301595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:10.302596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:10.303486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:10.303486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:10.303486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:10.303486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:10.303486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:10.303486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:10.303486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:10.303486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:10.308169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 342/680] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:10.316020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec6f23-c88a-4339-95f3-63844691d296 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:10.344758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:10.441497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1f1b4-4e11-46e7-8b30-1df191f6ddcd None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:10.457744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec6f23-c88a-4339-95f3-63844691d296 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:10.482688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641f2dc-7685-4e00-8b31-ca382d2e1bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:10.484748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641f2dc-7685-4e00-8b31-ca382d2e1bea None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:10.484748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641f2dc-7685-4e00-8b31-ca382d2e1bea None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:10.484748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641f2dc-7685-4e00-8b31-ca382d2e1bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:10.498806 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641f2dc-7685-4e00-8b31-ca382d2e1bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:10.500515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641f2dc-7685-4e00-8b31-ca382d2e1bea None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:10.501221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6412464-0846-4a8c-b456-d6231a78dd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:10.501832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6412464-0846-4a8c-b456-d6231a78dd72 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:10.502106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6412464-0846-4a8c-b456-d6231a78dd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:10.502377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6412464-0846-4a8c-b456-d6231a78dd72 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:10.503074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6412464-0846-4a8c-b456-d6231a78dd72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:10.503377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6412464-0846-4a8c-b456-d6231a78dd72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:10.504828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6412464-0846-4a8c-b456-d6231a78dd72 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-c641f2dc-7685-4e00-8b31-ca382d2e1bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:10.517178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:10.518329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:10.519449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:10.519449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:10.519449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:10.523597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 339/681] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:10.523597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641f2dc-7685-4e00-8b31-ca382d2e1bea None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:10.528887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 343/682] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:10.529741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6412464-0846-4a8c-b456-d6231a78dd72 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:10.577273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:10.612036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:10.635833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641f2dc-7685-4e00-8b31-ca382d2e1bea None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:10.677653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6412464-0846-4a8c-b456-d6231a78dd72 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:10.681502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b376e-66c7-4341-bb76-6316ac20865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:10.682387 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b376e-66c7-4341-bb76-6316ac20865d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:10.682886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b376e-66c7-4341-bb76-6316ac20865d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:10.683359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b376e-66c7-4341-bb76-6316ac20865d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:10.698018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b376e-66c7-4341-bb76-6316ac20865d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-e12b376e-66c7-4341-bb76-6316ac20865d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:10.704851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:10.705790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:10.706651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 340/683] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:10.707360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b376e-66c7-4341-bb76-6316ac20865d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:10.727628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66168dda-47e6-4c10-acba-88d5fdc4250b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:10.728655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66168dda-47e6-4c10-acba-88d5fdc4250b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:10.729145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66168dda-47e6-4c10-acba-88d5fdc4250b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:10.729626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66168dda-47e6-4c10-acba-88d5fdc4250b None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:10.730472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66168dda-47e6-4c10-acba-88d5fdc4250b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:10.731055 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66168dda-47e6-4c10-acba-88d5fdc4250b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:10.732781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66168dda-47e6-4c10-acba-88d5fdc4250b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:10.746778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 344/684] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:10.748388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66168dda-47e6-4c10-acba-88d5fdc4250b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:10.823493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b376e-66c7-4341-bb76-6316ac20865d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:10.857195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:10.859265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e3478-04dc-4441-a3c2-165c880dc0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:10.859907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e3478-04dc-4441-a3c2-165c880dc0ef None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:10.860188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e3478-04dc-4441-a3c2-165c880dc0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:10.860459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e3478-04dc-4441-a3c2-165c880dc0ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:10.861183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e3478-04dc-4441-a3c2-165c880dc0ef None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=9a89a2e6aeb4495a84a61bd82b02bfc4) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:10.861508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e3478-04dc-4441-a3c2-165c880dc0ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:10.863021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e3478-04dc-4441-a3c2-165c880dc0ef None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:10.869741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66168dda-47e6-4c10-acba-88d5fdc4250b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:10.875972 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 341/685] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:10 2022] GET /identity/v3/role_assignments?user.id=9a89a2e6aeb4495a84a61bd82b02bfc4&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:10.876930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e3478-04dc-4441-a3c2-165c880dc0ef None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:10.879908 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:10.914111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ff0e0-8b56-4baa-9e54-ef2f0104e17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:10.915160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ff0e0-8b56-4baa-9e54-ef2f0104e17e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:10.915707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ff0e0-8b56-4baa-9e54-ef2f0104e17e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:10.916187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ff0e0-8b56-4baa-9e54-ef2f0104e17e None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:10.917111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ff0e0-8b56-4baa-9e54-ef2f0104e17e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:10.917718 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ff0e0-8b56-4baa-9e54-ef2f0104e17e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:10.919387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ff0e0-8b56-4baa-9e54-ef2f0104e17e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:10.930008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 345/686] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:10.936320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ff0e0-8b56-4baa-9e54-ef2f0104e17e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:11.095569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ff0e0-8b56-4baa-9e54-ef2f0104e17e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:11.095898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e3478-04dc-4441-a3c2-165c880dc0ef None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:11.140590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f694d9-3e7a-4b15-a79e-9715c0d2f341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:11.141425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f694d9-3e7a-4b15-a79e-9715c0d2f341 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:11.141803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f694d9-3e7a-4b15-a79e-9715c0d2f341 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:11.142181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f694d9-3e7a-4b15-a79e-9715c0d2f341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:11.142628 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d3739-5287-4768-9e89-d675e39251ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:11.143249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3739-5287-4768-9e89-d675e39251ce None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:11.143524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3739-5287-4768-9e89-d675e39251ce None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:11.143859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3739-5287-4768-9e89-d675e39251ce None admin] PATH_INFO: `/v3/users/swift` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:11.152772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d3739-5287-4768-9e89-d675e39251ce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:11.154433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d3739-5287-4768-9e89-d675e39251ce None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:11.155450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f694d9-3e7a-4b15-a79e-9715c0d2f341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:11.157124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f694d9-3e7a-4b15-a79e-9715c0d2f341 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-7b5d3739-5287-4768-9e89-d675e39251ce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:11.167306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:11.168300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:11.169252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 342/687] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e1f694d9-3e7a-4b15-a79e-9715c0d2f341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:11.176457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:11.178869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:11.181019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:11.181019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:11.181019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:11.181019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 346/688] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:11.181549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d3739-5287-4768-9e89-d675e39251ce None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:11.184460 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f694d9-3e7a-4b15-a79e-9715c0d2f341 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:11.199902 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:11.270974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:11.338529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f694d9-3e7a-4b15-a79e-9715c0d2f341 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:11.342839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d3739-5287-4768-9e89-d675e39251ce None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:11.398967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21058f34-7bcc-4738-8f30-7a87cef51ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:11.399703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21058f34-7bcc-4738-8f30-7a87cef51ebb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:11.403558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21058f34-7bcc-4738-8f30-7a87cef51ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:11.403558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21058f34-7bcc-4738-8f30-7a87cef51ebb None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:11.403558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21058f34-7bcc-4738-8f30-7a87cef51ebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:11.403558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21058f34-7bcc-4738-8f30-7a87cef51ebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:11.403558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21058f34-7bcc-4738-8f30-7a87cef51ebb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:11.408284 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd274-3061-4372-ab52-5239f5866511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:11.408907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd274-3061-4372-ab52-5239f5866511 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:11.409182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd274-3061-4372-ab52-5239f5866511 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:11.409450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd274-3061-4372-ab52-5239f5866511 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:11.410178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dd274-3061-4372-ab52-5239f5866511 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:11.410476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dd274-3061-4372-ab52-5239f5866511 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:11.411969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dd274-3061-4372-ab52-5239f5866511 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:11.418183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 347/689] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:11.423128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21058f34-7bcc-4738-8f30-7a87cef51ebb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:11.444742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 343/690] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Sep 5 12:10:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:11.451283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd274-3061-4372-ab52-5239f5866511 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:11.537874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:11.577875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21058f34-7bcc-4738-8f30-7a87cef51ebb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:11.586833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd274-3061-4372-ab52-5239f5866511 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:11.610532 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:11.613600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26363723-0ae1-4a53-a6aa-53b256cd952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:11.614240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26363723-0ae1-4a53-a6aa-53b256cd952d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:11.614507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26363723-0ae1-4a53-a6aa-53b256cd952d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:11.614780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26363723-0ae1-4a53-a6aa-53b256cd952d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:11.622403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26363723-0ae1-4a53-a6aa-53b256cd952d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:11.624440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26363723-0ae1-4a53-a6aa-53b256cd952d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:11.628666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb13094-d610-4fa8-8fec-1182fca2cbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:11.629560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb13094-d610-4fa8-8fec-1182fca2cbc7 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:11.630043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb13094-d610-4fa8-8fec-1182fca2cbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:11.630520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb13094-d610-4fa8-8fec-1182fca2cbc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-26363723-0ae1-4a53-a6aa-53b256cd952d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:11.637282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:11.638323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:11.639278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 348/691] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Sep 5 12:10:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:11.644228 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26363723-0ae1-4a53-a6aa-53b256cd952d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:11.725441 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb13094-d610-4fa8-8fec-1182fca2cbc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:11.727830 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb13094-d610-4fa8-8fec-1182fca2cbc7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:11.755130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26363723-0ae1-4a53-a6aa-53b256cd952d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:11.780112 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:11.791313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aca4f-4aca-4cbc-a76a-238f3e072ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:11.791747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aca4f-4aca-4cbc-a76a-238f3e072ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:11.792030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aca4f-4aca-4cbc-a76a-238f3e072ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:11.792304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aca4f-4aca-4cbc-a76a-238f3e072ca3 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:11.793024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aca4f-4aca-4cbc-a76a-238f3e072ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:11.797560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aca4f-4aca-4cbc-a76a-238f3e072ca3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:11.797560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aca4f-4aca-4cbc-a76a-238f3e072ca3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:11.819062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 349/692] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Sep 5 12:10:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:11.820447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3082c-6294-4eb1-bab1-b4aac7e812e4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:11.820736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3082c-6294-4eb1-bab1-b4aac7e812e4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:11.821010 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3082c-6294-4eb1-bab1-b4aac7e812e4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:11.822537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 350/693] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:11.823242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3082c-6294-4eb1-bab1-b4aac7e812e4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:11.867723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 344/694] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:10:11.869280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dbeaf-d30f-47a9-94ba-04c581664349 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:11.869788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dbeaf-d30f-47a9-94ba-04c581664349 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:11.870264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dbeaf-d30f-47a9-94ba-04c581664349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:11.894408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ee9dbeaf-d30f-47a9-94ba-04c581664349 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:11.896655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ee9dbeaf-d30f-47a9-94ba-04c581664349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:11.921719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:11.940887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3082c-6294-4eb1-bab1-b4aac7e812e4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:11.985204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 345/695] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:11.986381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9dbeaf-d30f-47a9-94ba-04c581664349 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:11.987487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129df4ad-2f1c-4139-b3ed-e77cabe3b00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:11.988370 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129df4ad-2f1c-4139-b3ed-e77cabe3b00f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:11.989144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129df4ad-2f1c-4139-b3ed-e77cabe3b00f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:11.990578 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129df4ad-2f1c-4139-b3ed-e77cabe3b00f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.075066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:12.099937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9dbeaf-d30f-47a9-94ba-04c581664349 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:12.116225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129df4ad-2f1c-4139-b3ed-e77cabe3b00f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:12.117977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129df4ad-2f1c-4139-b3ed-e77cabe3b00f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:12.138515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d1faa-b25d-4612-b04d-ba8e43261443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:12.140229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d1faa-b25d-4612-b04d-ba8e43261443 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:12.140660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d1faa-b25d-4612-b04d-ba8e43261443 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:12.142791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d1faa-b25d-4612-b04d-ba8e43261443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.155305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d1faa-b25d-4612-b04d-ba8e43261443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:12.156849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d1faa-b25d-4612-b04d-ba8e43261443 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-bd1d1faa-b25d-4612-b04d-ba8e43261443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:12.174357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:12.175347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:12.176271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:12.176271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:12.176271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:12.180351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 346/696] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:12.181833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d1faa-b25d-4612-b04d-ba8e43261443 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:12.185114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:12.272187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 351/697] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 12:10:12.277539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129df4ad-2f1c-4139-b3ed-e77cabe3b00f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:12.351958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d1faa-b25d-4612-b04d-ba8e43261443 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:12.364335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:12.384734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129df4ad-2f1c-4139-b3ed-e77cabe3b00f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:12.409519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9fc7a-ef30-47ca-8d0c-c9886217df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:12.410632 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9fc7a-ef30-47ca-8d0c-c9886217df3c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:12.411065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9fc7a-ef30-47ca-8d0c-c9886217df3c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:12.411716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9fc7a-ef30-47ca-8d0c-c9886217df3c None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.412848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9fc7a-ef30-47ca-8d0c-c9886217df3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:12.413438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9fc7a-ef30-47ca-8d0c-c9886217df3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:12.415236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9fc7a-ef30-47ca-8d0c-c9886217df3c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:12.425001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd957aa-99c5-48da-8eeb-381d78f69c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:12.425389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd957aa-99c5-48da-8eeb-381d78f69c71 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:12.425772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd957aa-99c5-48da-8eeb-381d78f69c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:12.426135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd957aa-99c5-48da-8eeb-381d78f69c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 347/698] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:12.432293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9fc7a-ef30-47ca-8d0c-c9886217df3c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:12.442725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd957aa-99c5-48da-8eeb-381d78f69c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:12.444403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd957aa-99c5-48da-8eeb-381d78f69c71 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-7fd957aa-99c5-48da-8eeb-381d78f69c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:12.465980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:12.467201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:12.468424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:12.468424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:12.468424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:12.470732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 352/699] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:12.471722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd957aa-99c5-48da-8eeb-381d78f69c71 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:12.513518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:12.558501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9fc7a-ef30-47ca-8d0c-c9886217df3c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:12.581752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd957aa-99c5-48da-8eeb-381d78f69c71 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:12.595674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd8b3d-0c34-4856-9125-2fa68b423b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:12.596301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd8b3d-0c34-4856-9125-2fa68b423b3a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:12.596581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd8b3d-0c34-4856-9125-2fa68b423b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:12.596853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd8b3d-0c34-4856-9125-2fa68b423b3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.607240 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd8b3d-0c34-4856-9125-2fa68b423b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:12.609277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd8b3d-0c34-4856-9125-2fa68b423b3a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:12.618155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06112ff-3d25-4e26-b90b-f2fc21e5d263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:12.618809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06112ff-3d25-4e26-b90b-f2fc21e5d263 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:12.619096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06112ff-3d25-4e26-b90b-f2fc21e5d263 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:12.619370 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06112ff-3d25-4e26-b90b-f2fc21e5d263 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.621458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-4abd8b3d-0c34-4856-9125-2fa68b423b3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:12.625575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:12.626493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:12.627375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:12.627375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:12.627375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:12.627375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 348/700] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:12.627641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd8b3d-0c34-4856-9125-2fa68b423b3a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:12.632370 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06112ff-3d25-4e26-b90b-f2fc21e5d263 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:12.633847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06112ff-3d25-4e26-b90b-f2fc21e5d263 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f06112ff-3d25-4e26-b90b-f2fc21e5d263 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:12.646615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:12.647641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:12.648528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:12.648528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:12.648528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:12.648528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:12.648528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:12.648528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:12.648528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:12.648528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:12.648528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 353/701] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:12.649511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac881f36-cf9a-4f1b-9186-4178066a3597 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:12.649511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac881f36-cf9a-4f1b-9186-4178066a3597 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:12.649511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac881f36-cf9a-4f1b-9186-4178066a3597 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.654929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 354/702] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:12.654929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac881f36-cf9a-4f1b-9186-4178066a3597 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:12.750654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd8b3d-0c34-4856-9125-2fa68b423b3a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:12.762934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac881f36-cf9a-4f1b-9186-4178066a3597 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:12.767186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:12.789237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8a349-3072-45ef-985a-482ed29a22e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:12.790249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8a349-3072-45ef-985a-482ed29a22e6 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:12.790706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8a349-3072-45ef-985a-482ed29a22e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:12.791153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8a349-3072-45ef-985a-482ed29a22e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.792139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8a349-3072-45ef-985a-482ed29a22e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:12.792745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8a349-3072-45ef-985a-482ed29a22e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:12.794121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8a349-3072-45ef-985a-482ed29a22e6 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:12.798326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f2371-7ed4-4e4d-9795-73b2d63175c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:12.798961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f2371-7ed4-4e4d-9795-73b2d63175c7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:12.805576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f2371-7ed4-4e4d-9795-73b2d63175c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:12.805576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f2371-7ed4-4e4d-9795-73b2d63175c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.805576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f2371-7ed4-4e4d-9795-73b2d63175c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:12.805576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f2371-7ed4-4e4d-9795-73b2d63175c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:12.805576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f2371-7ed4-4e4d-9795-73b2d63175c7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:12.813007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 349/703] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:12.813007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8a349-3072-45ef-985a-482ed29a22e6 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:12.818778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 355/704] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:12.820992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9aa20-d1fb-4a6b-86c5-ccdec366b996 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:12.821275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9aa20-d1fb-4a6b-86c5-ccdec366b996 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:12.821547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9aa20-d1fb-4a6b-86c5-ccdec366b996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.843212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2ce9aa20-d1fb-4a6b-86c5-ccdec366b996 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:12.844415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2ce9aa20-d1fb-4a6b-86c5-ccdec366b996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:12.898180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:12.917988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 356/705] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:12.918797 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9aa20-d1fb-4a6b-86c5-ccdec366b996 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:12.922875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8a349-3072-45ef-985a-482ed29a22e6 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:12.960518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e2f87-661f-455a-a53f-7d2f6cb150af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:12.961188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e2f87-661f-455a-a53f-7d2f6cb150af None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:12.961527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e2f87-661f-455a-a53f-7d2f6cb150af None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:12.961806 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e2f87-661f-455a-a53f-7d2f6cb150af None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:12.962503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e2f87-661f-455a-a53f-7d2f6cb150af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:12.962808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e2f87-661f-455a-a53f-7d2f6cb150af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:12.965153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e2f87-661f-455a-a53f-7d2f6cb150af None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:12.982384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 350/706] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:12.982384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e2f87-661f-455a-a53f-7d2f6cb150af None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:13.040019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:13.058590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9aa20-d1fb-4a6b-86c5-ccdec366b996 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:13.100696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dee5b-832c-4048-a70d-b4c43f2f5b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:13.101377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dee5b-832c-4048-a70d-b4c43f2f5b47 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:13.101606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dee5b-832c-4048-a70d-b4c43f2f5b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:13.101880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dee5b-832c-4048-a70d-b4c43f2f5b47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:13.109072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dee5b-832c-4048-a70d-b4c43f2f5b47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:13.110483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dee5b-832c-4048-a70d-b4c43f2f5b47 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:13.114891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e2f87-661f-455a-a53f-7d2f6cb150af None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-4e6dee5b-832c-4048-a70d-b4c43f2f5b47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:13.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:13.122955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:13.123928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:13.123928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:13.123928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:13.123928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:13.123928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:13.123928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:13.123928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:13.123928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:13.123928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 357/707] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:13.124410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dee5b-832c-4048-a70d-b4c43f2f5b47 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:13.163955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2edc8f-9700-4a9d-a94c-3ce8ba9c04ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:13.164903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2edc8f-9700-4a9d-a94c-3ce8ba9c04ef None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:13.165395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2edc8f-9700-4a9d-a94c-3ce8ba9c04ef None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:13.165878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2edc8f-9700-4a9d-a94c-3ce8ba9c04ef None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:13.166715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2edc8f-9700-4a9d-a94c-3ce8ba9c04ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:13.168925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2edc8f-9700-4a9d-a94c-3ce8ba9c04ef None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:13.173804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:13.230324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dee5b-832c-4048-a70d-b4c43f2f5b47 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:13.263125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67d779-792e-4999-95fc-cdd43085e32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:13.263765 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d779-792e-4999-95fc-cdd43085e32e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:13.264046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d779-792e-4999-95fc-cdd43085e32e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:13.264318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d779-792e-4999-95fc-cdd43085e32e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:13.265573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67d779-792e-4999-95fc-cdd43085e32e None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=b48aa41ff66b42329926bc42379d9800) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:13.265879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67d779-792e-4999-95fc-cdd43085e32e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:13.267277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67d779-792e-4999-95fc-cdd43085e32e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:13.272929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 358/708] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:13 2022] GET /identity/v3/role_assignments?user.id=b48aa41ff66b42329926bc42379d9800&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:13.273695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67d779-792e-4999-95fc-cdd43085e32e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:13.304777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 351/709] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:10:12 2022] POST /identity/v3/users => generated 254 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:13.305635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2edc8f-9700-4a9d-a94c-3ce8ba9c04ef None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:13.401921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67d779-792e-4999-95fc-cdd43085e32e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:13.412766 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:13.449873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:13.455578 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b96e3c-3e69-49fc-b124-b280a9d7a16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:13.455901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96e3c-3e69-49fc-b124-b280a9d7a16a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:13.457516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96e3c-3e69-49fc-b124-b280a9d7a16a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:13.457516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96e3c-3e69-49fc-b124-b280a9d7a16a None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:13.457516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b96e3c-3e69-49fc-b124-b280a9d7a16a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:13.457913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b96e3c-3e69-49fc-b124-b280a9d7a16a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:13.460169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b96e3c-3e69-49fc-b124-b280a9d7a16a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:13.464557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2edc8f-9700-4a9d-a94c-3ce8ba9c04ef None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:13.477899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 359/710] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:13.479497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146f022-9503-45aa-8027-1b0ae267874e None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:13.480050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146f022-9503-45aa-8027-1b0ae267874e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:13.480537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146f022-9503-45aa-8027-1b0ae267874e None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:13.482285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 360/711] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:13.485939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146f022-9503-45aa-8027-1b0ae267874e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:13.513968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a72bff-7b88-425f-9443-1c36dd9ed1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:13.514518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72bff-7b88-425f-9443-1c36dd9ed1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:13.515795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72bff-7b88-425f-9443-1c36dd9ed1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:13.515795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72bff-7b88-425f-9443-1c36dd9ed1f5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:13.516524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a72bff-7b88-425f-9443-1c36dd9ed1f5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:13.519085 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a72bff-7b88-425f-9443-1c36dd9ed1f5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-23a72bff-7b88-425f-9443-1c36dd9ed1f5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:13.533311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:13.534457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:10:13.535396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:13.536217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 352/712] 10.222.0.62 () {56 vars in 1129 bytes} [Mon Sep 5 12:10:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:13.536889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504cd14-0525-4237-992d-3e6fa3a95f62 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:13.537983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504cd14-0525-4237-992d-3e6fa3a95f62 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:13.537983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504cd14-0525-4237-992d-3e6fa3a95f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:13.559709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-9504cd14-0525-4237-992d-3e6fa3a95f62 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:13.561930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9504cd14-0525-4237-992d-3e6fa3a95f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:13.618750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146f022-9503-45aa-8027-1b0ae267874e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:13.650346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 353/713] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:13.652051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504cd14-0525-4237-992d-3e6fa3a95f62 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:13.656570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfee8ad-d35e-41cc-8c36-45fda936acfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:13.657281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfee8ad-d35e-41cc-8c36-45fda936acfd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:13.657650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfee8ad-d35e-41cc-8c36-45fda936acfd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:13.658050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfee8ad-d35e-41cc-8c36-45fda936acfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:13.658887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfee8ad-d35e-41cc-8c36-45fda936acfd None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=7956a7748ec94a618eb466f5536220b0) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:13.659290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfee8ad-d35e-41cc-8c36-45fda936acfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:13.660958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfee8ad-d35e-41cc-8c36-45fda936acfd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:13.667483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 361/714] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:13 2022] GET /identity/v3/role_assignments?user.id=7956a7748ec94a618eb466f5536220b0&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=705907b9bdec47cebf88b868a6a56356 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:13.797927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:13.821731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504cd14-0525-4237-992d-3e6fa3a95f62 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:13.861901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569511e-67ce-4ae7-b2d9-c60af3b3bb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:13.862866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569511e-67ce-4ae7-b2d9-c60af3b3bb1b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:13.863376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569511e-67ce-4ae7-b2d9-c60af3b3bb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:13.863892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569511e-67ce-4ae7-b2d9-c60af3b3bb1b None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:13.864723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569511e-67ce-4ae7-b2d9-c60af3b3bb1b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:13.865242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569511e-67ce-4ae7-b2d9-c60af3b3bb1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:13.866813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569511e-67ce-4ae7-b2d9-c60af3b3bb1b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:13.871954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 354/715] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Sep 5 12:10:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:13.878598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfee8ad-d35e-41cc-8c36-45fda936acfd None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:13.931905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:14.000181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569511e-67ce-4ae7-b2d9-c60af3b3bb1b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:14.028206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfee8ad-d35e-41cc-8c36-45fda936acfd None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:14.108338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021f82e-7dd3-4361-893d-165af467e07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:14.109342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021f82e-7dd3-4361-893d-165af467e07b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:14.109827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021f82e-7dd3-4361-893d-165af467e07b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:14.110323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021f82e-7dd3-4361-893d-165af467e07b None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:14.111181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021f82e-7dd3-4361-893d-165af467e07b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:14.111713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021f82e-7dd3-4361-893d-165af467e07b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:14.113301 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021f82e-7dd3-4361-893d-165af467e07b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:14.128406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 362/716] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Sep 5 12:10:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:14.146982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569511e-67ce-4ae7-b2d9-c60af3b3bb1b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:14.184504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989e388-9b60-402a-a04c-e2b51899695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:14.185579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989e388-9b60-402a-a04c-e2b51899695d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:14.186070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989e388-9b60-402a-a04c-e2b51899695d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:14.187102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989e388-9b60-402a-a04c-e2b51899695d None admin] PATH_INFO: `/v3/services/network` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:14.187924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989e388-9b60-402a-a04c-e2b51899695d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:14.189538 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989e388-9b60-402a-a04c-e2b51899695d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-6989e388-9b60-402a-a04c-e2b51899695d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:14.199241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:14.200297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:10:14.201195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:14.204463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 355/717] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:13 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:14.211691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021f82e-7dd3-4361-893d-165af467e07b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:14.376777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:14.382460 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021f82e-7dd3-4361-893d-165af467e07b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:14.419290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e3d00-73ba-4923-8008-a8f51044e7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:14.419932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e3d00-73ba-4923-8008-a8f51044e7ff None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:14.420208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e3d00-73ba-4923-8008-a8f51044e7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:14.420480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e3d00-73ba-4923-8008-a8f51044e7ff None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:14.421125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e3d00-73ba-4923-8008-a8f51044e7ff None admin] RBAC: Adding query filter params (name=network) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:14.421422 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e3d00-73ba-4923-8008-a8f51044e7ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:14.422857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e3d00-73ba-4923-8008-a8f51044e7ff None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:14.427421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 363/718] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:14.432639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989e388-9b60-402a-a04c-e2b51899695d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:14.448666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:14.588272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989e388-9b60-402a-a04c-e2b51899695d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:14.637730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795eb958-3e26-487d-8e82-4ec81271ae0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:14.638266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eb958-3e26-487d-8e82-4ec81271ae0b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:14.638542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eb958-3e26-487d-8e82-4ec81271ae0b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:14.638807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eb958-3e26-487d-8e82-4ec81271ae0b None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:14.639470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795eb958-3e26-487d-8e82-4ec81271ae0b None admin] RBAC: Adding query filter params (type=network) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:14.639814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795eb958-3e26-487d-8e82-4ec81271ae0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:14.641183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795eb958-3e26-487d-8e82-4ec81271ae0b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:14.653068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 356/719] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:15.703451 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83969826-cb0b-45dd-a529-7bd0c6212469 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:15.704506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83969826-cb0b-45dd-a529-7bd0c6212469 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:15.705003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83969826-cb0b-45dd-a529-7bd0c6212469 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:15.706820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 364/720] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:15.711707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7d1cf-d58f-48cd-a8c2-79a96068758b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:15.712112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7d1cf-d58f-48cd-a8c2-79a96068758b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:15.712284 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7d1cf-d58f-48cd-a8c2-79a96068758b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:15.735172 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ddb7d1cf-d58f-48cd-a8c2-79a96068758b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:15.740451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ddb7d1cf-d58f-48cd-a8c2-79a96068758b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:15.829946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 357/721] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:15.851921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a871362-6e72-4b8b-8e05-86d787fd55eb None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:15.852239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a871362-6e72-4b8b-8e05-86d787fd55eb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:15.852480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a871362-6e72-4b8b-8e05-86d787fd55eb None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:15.854023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 365/722] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:15.859511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f141c-0f44-4c30-bfae-59658b10fca5 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:15.859847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f141c-0f44-4c30-bfae-59658b10fca5 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:15.860128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f141c-0f44-4c30-bfae-59658b10fca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:15.872827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:15.885021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-686f141c-0f44-4c30-bfae-59658b10fca5 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:15.886222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-686f141c-0f44-4c30-bfae-59658b10fca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:15.925778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde7e4-3627-400d-8ffa-bf71c2c869f7 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:15.926051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde7e4-3627-400d-8ffa-bf71c2c869f7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:15.926307 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde7e4-3627-400d-8ffa-bf71c2c869f7 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:15.927907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 366/723] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:15.932761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac94497-306c-4ccb-8aa3-fa6fd71dc04b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:15.933095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac94497-306c-4ccb-8aa3-fa6fd71dc04b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:15.933378 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac94497-306c-4ccb-8aa3-fa6fd71dc04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:15.968516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-aac94497-306c-4ccb-8aa3-fa6fd71dc04b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:15.969980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-aac94497-306c-4ccb-8aa3-fa6fd71dc04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:15.993931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 358/724] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 12:10:16.107141 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 367/725] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:16.255314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f141c-0f44-4c30-bfae-59658b10fca5 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:16.296792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac94497-306c-4ccb-8aa3-fa6fd71dc04b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:16.402132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:16.413075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac94497-306c-4ccb-8aa3-fa6fd71dc04b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:16.426611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f141c-0f44-4c30-bfae-59658b10fca5 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:16.458232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e7963-df28-4a05-bb5d-d1f84bddd86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:16.458976 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e7963-df28-4a05-bb5d-d1f84bddd86d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:16.459574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e7963-df28-4a05-bb5d-d1f84bddd86d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:16.459695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e7963-df28-4a05-bb5d-d1f84bddd86d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:16.465318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6f80c-2c1c-4318-b5ae-45e96f48dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:16.465940 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6f80c-2c1c-4318-b5ae-45e96f48dc38 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:16.466214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6f80c-2c1c-4318-b5ae-45e96f48dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:16.466478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6f80c-2c1c-4318-b5ae-45e96f48dc38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:16.469685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:16.471444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e7963-df28-4a05-bb5d-d1f84bddd86d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-c94e7963-df28-4a05-bb5d-d1f84bddd86d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:16.476448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:16.484572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:16.485461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:16.486236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6f80c-2c1c-4318-b5ae-45e96f48dc38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:16.486325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 368/726] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:16.486325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6808ab-3cca-42e7-994b-ee9c55bdb712 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:16.486325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6808ab-3cca-42e7-994b-ee9c55bdb712 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:16.486325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6808ab-3cca-42e7-994b-ee9c55bdb712 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:16.486325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 369/727] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:16.486325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6808ab-3cca-42e7-994b-ee9c55bdb712 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-85f6f80c-2c1c-4318-b5ae-45e96f48dc38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:16.486689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:16.488375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 359/728] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:16.491579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6f80c-2c1c-4318-b5ae-45e96f48dc38 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:16.591549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6f80c-2c1c-4318-b5ae-45e96f48dc38 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:16.600293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6808ab-3cca-42e7-994b-ee9c55bdb712 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:16.627140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7551e8-9e1f-4521-9641-f6e245f20a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:16.627783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7551e8-9e1f-4521-9641-f6e245f20a29 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:16.628061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7551e8-9e1f-4521-9641-f6e245f20a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:16.628336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7551e8-9e1f-4521-9641-f6e245f20a29 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:16.628993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7551e8-9e1f-4521-9641-f6e245f20a29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:16.629297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7551e8-9e1f-4521-9641-f6e245f20a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:16.630667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7551e8-9e1f-4521-9641-f6e245f20a29 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:16.636573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c807bd3-4f92-4a83-87c5-e8ff36e626f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:16.637258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c807bd3-4f92-4a83-87c5-e8ff36e626f3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:16.637467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c807bd3-4f92-4a83-87c5-e8ff36e626f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:16.637735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c807bd3-4f92-4a83-87c5-e8ff36e626f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:16.643584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 360/729] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:16.643584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7551e8-9e1f-4521-9641-f6e245f20a29 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:16.653546 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c807bd3-4f92-4a83-87c5-e8ff36e626f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-3c807bd3-4f92-4a83-87c5-e8ff36e626f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:16.659331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:16.660427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:16.661268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 370/730] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:16.662412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2ea7-7c25-4f08-91d5-b8d3b3f72188 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:16.662412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2ea7-7c25-4f08-91d5-b8d3b3f72188 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:16.662412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2ea7-7c25-4f08-91d5-b8d3b3f72188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:16.681342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-778e2ea7-7c25-4f08-91d5-b8d3b3f72188 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:16.682602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-778e2ea7-7c25-4f08-91d5-b8d3b3f72188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:16.686356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:16.708684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:16.759746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 371/731] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:16.759746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e2ea7-7c25-4f08-91d5-b8d3b3f72188 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:16.760355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7551e8-9e1f-4521-9641-f6e245f20a29 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:16.796141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782f4a4-faaf-4fbe-b911-89b9a942f9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:16.796841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782f4a4-faaf-4fbe-b911-89b9a942f9fd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:16.797310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782f4a4-faaf-4fbe-b911-89b9a942f9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:16.797650 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782f4a4-faaf-4fbe-b911-89b9a942f9fd None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:16.798359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782f4a4-faaf-4fbe-b911-89b9a942f9fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:16.798723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782f4a4-faaf-4fbe-b911-89b9a942f9fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:16.800297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782f4a4-faaf-4fbe-b911-89b9a942f9fd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:16.813581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 361/732] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:10:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:16.814537 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782f4a4-faaf-4fbe-b911-89b9a942f9fd None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:16.893276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e2ea7-7c25-4f08-91d5-b8d3b3f72188 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:16.925126 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782f4a4-faaf-4fbe-b911-89b9a942f9fd None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:16.939728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde4b0c-06fb-4453-a5d3-eddb865818fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:16.940937 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde4b0c-06fb-4453-a5d3-eddb865818fa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:16.941703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde4b0c-06fb-4453-a5d3-eddb865818fa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:16.942207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde4b0c-06fb-4453-a5d3-eddb865818fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:16.945904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:16.955341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:16.958549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde4b0c-06fb-4453-a5d3-eddb865818fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:16.960445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde4b0c-06fb-4453-a5d3-eddb865818fa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:16.961013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2ba5c-a4d1-44c9-a810-97159a8996c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:16.961734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2ba5c-a4d1-44c9-a810-97159a8996c4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:16.962160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2ba5c-a4d1-44c9-a810-97159a8996c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:16.962522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2ba5c-a4d1-44c9-a810-97159a8996c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:16.963481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2ba5c-a4d1-44c9-a810-97159a8996c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:16.963899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2ba5c-a4d1-44c9-a810-97159a8996c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:16.967098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2ba5c-a4d1-44c9-a810-97159a8996c4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:16.985938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 362/733] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:16.985938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2ba5c-a4d1-44c9-a810-97159a8996c4 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-edde4b0c-06fb-4453-a5d3-eddb865818fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:16.992975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:16.993979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:16.994934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:16.994934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:16.994934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:16.994934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 372/734] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:16.995162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde4b0c-06fb-4453-a5d3-eddb865818fa None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:17.103012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2ba5c-a4d1-44c9-a810-97159a8996c4 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:17.116190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde4b0c-06fb-4453-a5d3-eddb865818fa None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:17.138941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c0938-36fd-4bf4-ab18-efce63c92419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:17.140231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c0938-36fd-4bf4-ab18-efce63c92419 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.140528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c0938-36fd-4bf4-ab18-efce63c92419 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.140801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c0938-36fd-4bf4-ab18-efce63c92419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.150758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c0938-36fd-4bf4-ab18-efce63c92419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:17.150893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71102f62-793e-474d-9626-e92b7e917859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:17.151343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71102f62-793e-474d-9626-e92b7e917859 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.151653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71102f62-793e-474d-9626-e92b7e917859 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.151926 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71102f62-793e-474d-9626-e92b7e917859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.152171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c0938-36fd-4bf4-ab18-efce63c92419 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:17.162491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71102f62-793e-474d-9626-e92b7e917859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:17.163949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71102f62-793e-474d-9626-e92b7e917859 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-162c0938-36fd-4bf4-ab18-efce63c92419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:17.166543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:17.167517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:17.168479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:17.168479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:17.168479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:17.168479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 363/735] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:17.168479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c0938-36fd-4bf4-ab18-efce63c92419 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-71102f62-793e-474d-9626-e92b7e917859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:17.180297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:17.181239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:17.182135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:17.182135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:17.182135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:17.183676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 373/736] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:17.184641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71102f62-793e-474d-9626-e92b7e917859 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:17.206777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:17.229681 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:17.271729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c0938-36fd-4bf4-ab18-efce63c92419 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:17.282976 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71102f62-793e-474d-9626-e92b7e917859 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:17.304626 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451f16b-228c-48dc-bb0e-1d2df4bab7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:17.305234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451f16b-228c-48dc-bb0e-1d2df4bab7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.305514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451f16b-228c-48dc-bb0e-1d2df4bab7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.305784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451f16b-228c-48dc-bb0e-1d2df4bab7c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.306425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451f16b-228c-48dc-bb0e-1d2df4bab7c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:17.306729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451f16b-228c-48dc-bb0e-1d2df4bab7c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:17.308143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451f16b-228c-48dc-bb0e-1d2df4bab7c6 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:17.315674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f99519-1a2a-46a1-a61e-5826c8e7d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:17.316270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f99519-1a2a-46a1-a61e-5826c8e7d3a8 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.316547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f99519-1a2a-46a1-a61e-5826c8e7d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.316818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f99519-1a2a-46a1-a61e-5826c8e7d3a8 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.317452 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f99519-1a2a-46a1-a61e-5826c8e7d3a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:17.318789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f99519-1a2a-46a1-a61e-5826c8e7d3a8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:17.319741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 364/737] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:17.320907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a756a-79fd-43c9-9a04-09191a303636 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.321195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a756a-79fd-43c9-9a04-09191a303636 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.321474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a756a-79fd-43c9-9a04-09191a303636 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.326964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 365/738] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:17.326964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a756a-79fd-43c9-9a04-09191a303636 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:17.396835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 374/739] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:10:17 2022] POST /identity/v3/services => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:17.397662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f99519-1a2a-46a1-a61e-5826c8e7d3a8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:17.434341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:17.451199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a756a-79fd-43c9-9a04-09191a303636 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:17.492220 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5db4d-59eb-45a6-8e25-f3e32397cfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:17.492837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5db4d-59eb-45a6-8e25-f3e32397cfb1 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.493111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5db4d-59eb-45a6-8e25-f3e32397cfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.493395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5db4d-59eb-45a6-8e25-f3e32397cfb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.494022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5db4d-59eb-45a6-8e25-f3e32397cfb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:17.494318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5db4d-59eb-45a6-8e25-f3e32397cfb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:17.495749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5db4d-59eb-45a6-8e25-f3e32397cfb1 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:17.508690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 366/740] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:17.509584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5db4d-59eb-45a6-8e25-f3e32397cfb1 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:17.520823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f99519-1a2a-46a1-a61e-5826c8e7d3a8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:17.524675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:17.560273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e531960-a630-4b8f-8c1d-01ecddd1c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:17.560887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531960-a630-4b8f-8c1d-01ecddd1c382 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.561274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531960-a630-4b8f-8c1d-01ecddd1c382 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.562370 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531960-a630-4b8f-8c1d-01ecddd1c382 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.563021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e531960-a630-4b8f-8c1d-01ecddd1c382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:17.563327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e531960-a630-4b8f-8c1d-01ecddd1c382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:17.564794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e531960-a630-4b8f-8c1d-01ecddd1c382 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:17.577903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 375/741] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:17.579131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358843b4-2bf4-4d67-92c5-aa31cbe374db None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.579423 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358843b4-2bf4-4d67-92c5-aa31cbe374db None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.579707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358843b4-2bf4-4d67-92c5-aa31cbe374db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.603058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-358843b4-2bf4-4d67-92c5-aa31cbe374db None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:17.604319 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-358843b4-2bf4-4d67-92c5-aa31cbe374db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:17.633143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5db4d-59eb-45a6-8e25-f3e32397cfb1 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:17.670376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891638-7ed0-483e-ad45-566fa268e3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:17.671550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891638-7ed0-483e-ad45-566fa268e3fe None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.671830 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891638-7ed0-483e-ad45-566fa268e3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.672392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891638-7ed0-483e-ad45-566fa268e3fe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.676118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 376/742] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 12:10:17.676872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358843b4-2bf4-4d67-92c5-aa31cbe374db None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:17.690623 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69891638-7ed0-483e-ad45-566fa268e3fe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:17.693314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69891638-7ed0-483e-ad45-566fa268e3fe None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-69891638-7ed0-483e-ad45-566fa268e3fe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:17.709287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:17.710301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 367/743] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:17.711289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891638-7ed0-483e-ad45-566fa268e3fe None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:17.736066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:17.784920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:17.791787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358843b4-2bf4-4d67-92c5-aa31cbe374db None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:17.829389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cf442-3d8e-43ab-9abd-3ff4602e5f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:17.830123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cf442-3d8e-43ab-9abd-3ff4602e5f53 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.830459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cf442-3d8e-43ab-9abd-3ff4602e5f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.831072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cf442-3d8e-43ab-9abd-3ff4602e5f53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.838470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cf442-3d8e-43ab-9abd-3ff4602e5f53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:17.840352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cf442-3d8e-43ab-9abd-3ff4602e5f53 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f82cf442-3d8e-43ab-9abd-3ff4602e5f53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:17.852641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:17.853648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:17.854600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:17.854600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:17.854600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:17.854600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:17.854600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:17.854600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:17.854600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:17.854600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:17.854600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:17.854600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:17.859213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 377/744] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Sep 5 12:10:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:17.860209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cf442-3d8e-43ab-9abd-3ff4602e5f53 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:17.865832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891638-7ed0-483e-ad45-566fa268e3fe None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:17.911621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aee71-eb7e-4ad6-a854-963aa357079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:17.912391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aee71-eb7e-4ad6-a854-963aa357079b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:17.912778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aee71-eb7e-4ad6-a854-963aa357079b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:17.913053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aee71-eb7e-4ad6-a854-963aa357079b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:17.921681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7aee71-eb7e-4ad6-a854-963aa357079b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:17.923227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7aee71-eb7e-4ad6-a854-963aa357079b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-6d7aee71-eb7e-4ad6-a854-963aa357079b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:17.934671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:17.935677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:17.936666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 368/745] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Sep 5 12:10:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:17.937349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aee71-eb7e-4ad6-a854-963aa357079b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:17.986562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cf442-3d8e-43ab-9abd-3ff4602e5f53 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:18.013523 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:18.021361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092b17a-6ef4-4ac2-9734-36be3a0fd0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:18.021968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092b17a-6ef4-4ac2-9734-36be3a0fd0ce None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:18.022243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092b17a-6ef4-4ac2-9734-36be3a0fd0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:18.022518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092b17a-6ef4-4ac2-9734-36be3a0fd0ce None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:18.023233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092b17a-6ef4-4ac2-9734-36be3a0fd0ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:18.023529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092b17a-6ef4-4ac2-9734-36be3a0fd0ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:18.024961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092b17a-6ef4-4ac2-9734-36be3a0fd0ce None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:18.043279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 378/746] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Sep 5 12:10:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:18.048515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aee71-eb7e-4ad6-a854-963aa357079b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:18.049578 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092b17a-6ef4-4ac2-9734-36be3a0fd0ce None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:18.064647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:18.091802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f37875-c80a-4c0d-aa97-e25465876d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:18.092412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f37875-c80a-4c0d-aa97-e25465876d3f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:18.092689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f37875-c80a-4c0d-aa97-e25465876d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:18.093485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f37875-c80a-4c0d-aa97-e25465876d3f None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:18.094209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f37875-c80a-4c0d-aa97-e25465876d3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:18.094515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f37875-c80a-4c0d-aa97-e25465876d3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:18.095978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f37875-c80a-4c0d-aa97-e25465876d3f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:18.118931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 369/747] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Sep 5 12:10:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:18.118931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f37875-c80a-4c0d-aa97-e25465876d3f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:18.157408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092b17a-6ef4-4ac2-9734-36be3a0fd0ce None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:18.192710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851441aa-b640-4c07-9179-fc4d51c0e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:18.193320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851441aa-b640-4c07-9179-fc4d51c0e9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:18.194313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851441aa-b640-4c07-9179-fc4d51c0e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:18.194604 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851441aa-b640-4c07-9179-fc4d51c0e9e3 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:18.195330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851441aa-b640-4c07-9179-fc4d51c0e9e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:18.195666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851441aa-b640-4c07-9179-fc4d51c0e9e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:18.197081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851441aa-b640-4c07-9179-fc4d51c0e9e3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:18.221885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 379/748] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Sep 5 12:10:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:18.221885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851441aa-b640-4c07-9179-fc4d51c0e9e3 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:18.229148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f37875-c80a-4c0d-aa97-e25465876d3f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:18.255841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:18.263444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9590e-bcb1-4266-a84a-7af4a46d880a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:18.264073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9590e-bcb1-4266-a84a-7af4a46d880a None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:18.264424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9590e-bcb1-4266-a84a-7af4a46d880a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:18.265209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9590e-bcb1-4266-a84a-7af4a46d880a None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:18.265842 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9590e-bcb1-4266-a84a-7af4a46d880a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:18.267196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9590e-bcb1-4266-a84a-7af4a46d880a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:18.298793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:18.329807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851441aa-b640-4c07-9179-fc4d51c0e9e3 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:18.339667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 370/749] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:10:18 2022] POST /identity/v3/services => generated 240 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:18.339667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9590e-bcb1-4266-a84a-7af4a46d880a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:18.391079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508e131-5549-4f7d-acf7-3532208c7c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:18.393212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508e131-5549-4f7d-acf7-3532208c7c84 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:18.393957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508e131-5549-4f7d-acf7-3532208c7c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:18.394440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508e131-5549-4f7d-acf7-3532208c7c84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:18.484998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9590e-bcb1-4266-a84a-7af4a46d880a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:18.488982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508e131-5549-4f7d-acf7-3532208c7c84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:18.490511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508e131-5549-4f7d-acf7-3532208c7c84 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:18.523244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d3daf-91b3-43b4-b857-15a64d7a384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:18.523913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d3daf-91b3-43b4-b857-15a64d7a384e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:18.524163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d3daf-91b3-43b4-b857-15a64d7a384e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:18.524439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d3daf-91b3-43b4-b857-15a64d7a384e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:18.566766 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:18.613387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 380/750] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:10:18.614376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508e131-5549-4f7d-acf7-3532208c7c84 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:18.619382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d3daf-91b3-43b4-b857-15a64d7a384e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:18.621011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d3daf-91b3-43b4-b857-15a64d7a384e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:18.669987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:18.739988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508e131-5549-4f7d-acf7-3532208c7c84 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:18.753154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 371/751] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:10:18.754495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d3daf-91b3-43b4-b857-15a64d7a384e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:18.779816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca6844-3474-4514-919a-3f3e74b15e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:18.780718 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca6844-3474-4514-919a-3f3e74b15e53 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:18.781156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca6844-3474-4514-919a-3f3e74b15e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:18.781675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca6844-3474-4514-919a-3f3e74b15e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:18.793431 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca6844-3474-4514-919a-3f3e74b15e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:18.795523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca6844-3474-4514-919a-3f3e74b15e53 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-4eca6844-3474-4514-919a-3f3e74b15e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:18.812812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:18.813990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:18.815085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:18.815085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:18.815085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:18.816613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 381/752] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:18.817545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca6844-3474-4514-919a-3f3e74b15e53 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:18.850064 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:18.869231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:18.881566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d3daf-91b3-43b4-b857-15a64d7a384e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:18.924728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df55110-aeec-471b-920b-884fc827d571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:18.925828 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df55110-aeec-471b-920b-884fc827d571 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:18.926352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df55110-aeec-471b-920b-884fc827d571 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:18.926815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df55110-aeec-471b-920b-884fc827d571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:18.939394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df55110-aeec-471b-920b-884fc827d571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:18.941509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df55110-aeec-471b-920b-884fc827d571 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:18.941924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca6844-3474-4514-919a-3f3e74b15e53 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-5df55110-aeec-471b-920b-884fc827d571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:18.961999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:18.963033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:18.963939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:18.963939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:18.963939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:18.965464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 372/753] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:18.967114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df55110-aeec-471b-920b-884fc827d571 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:18.981215 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb2c58-a2bb-481a-b942-4b3a16d98b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:18.982165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb2c58-a2bb-481a-b942-4b3a16d98b25 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:18.982165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb2c58-a2bb-481a-b942-4b3a16d98b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:18.982463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb2c58-a2bb-481a-b942-4b3a16d98b25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:18.994344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb2c58-a2bb-481a-b942-4b3a16d98b25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:18.995858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb2c58-a2bb-481a-b942-4b3a16d98b25 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-ffeb2c58-a2bb-481a-b942-4b3a16d98b25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:19.011305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:19.012327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:19.013230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:19.013230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:19.013230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:19.013230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 382/754] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:19.013977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb2c58-a2bb-481a-b942-4b3a16d98b25 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:19.098261 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df55110-aeec-471b-920b-884fc827d571 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:19.109240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:19.135196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30021313-ad82-4709-a683-a036c75e6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:19.135939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30021313-ad82-4709-a683-a036c75e6b52 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.136293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30021313-ad82-4709-a683-a036c75e6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.136639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30021313-ad82-4709-a683-a036c75e6b52 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.137387 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30021313-ad82-4709-a683-a036c75e6b52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:19.137780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30021313-ad82-4709-a683-a036c75e6b52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:19.140117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30021313-ad82-4709-a683-a036c75e6b52 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.150457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:19.152627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb2c58-a2bb-481a-b942-4b3a16d98b25 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:19.158546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 373/755] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:19.158546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30021313-ad82-4709-a683-a036c75e6b52 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:19.195258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b81d0b-a85c-4e68-8f2b-b4db3dd23e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:19.195901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b81d0b-a85c-4e68-8f2b-b4db3dd23e7c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.196180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b81d0b-a85c-4e68-8f2b-b4db3dd23e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.196462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b81d0b-a85c-4e68-8f2b-b4db3dd23e7c None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.197104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b81d0b-a85c-4e68-8f2b-b4db3dd23e7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:19.197406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b81d0b-a85c-4e68-8f2b-b4db3dd23e7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:19.198760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b81d0b-a85c-4e68-8f2b-b4db3dd23e7c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.213047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 383/756] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:19.214009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b81d0b-a85c-4e68-8f2b-b4db3dd23e7c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:19.285900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30021313-ad82-4709-a683-a036c75e6b52 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:19.323322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25539692-01b0-4f2c-a5a2-616beba0e873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:19.323908 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25539692-01b0-4f2c-a5a2-616beba0e873 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.324172 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25539692-01b0-4f2c-a5a2-616beba0e873 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.324440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25539692-01b0-4f2c-a5a2-616beba0e873 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.325148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25539692-01b0-4f2c-a5a2-616beba0e873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:19.325451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25539692-01b0-4f2c-a5a2-616beba0e873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:19.326881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25539692-01b0-4f2c-a5a2-616beba0e873 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.338909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 374/757] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:19.340127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25539692-01b0-4f2c-a5a2-616beba0e873 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:19.346926 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b81d0b-a85c-4e68-8f2b-b4db3dd23e7c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:19.389557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:19.394587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a3549-7eea-4dc9-ac7c-a3e815f5e5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:19.395225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a3549-7eea-4dc9-ac7c-a3e815f5e5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.403810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a3549-7eea-4dc9-ac7c-a3e815f5e5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.403810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a3549-7eea-4dc9-ac7c-a3e815f5e5c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.403810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a3549-7eea-4dc9-ac7c-a3e815f5e5c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:19.404242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a3549-7eea-4dc9-ac7c-a3e815f5e5c8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.410619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-281a3549-7eea-4dc9-ac7c-a3e815f5e5c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:19.416717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:19.417635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:19.418533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:19.418533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:19.418533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:19.418533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:19.418533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:19.418533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:19.418533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:19.418533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:19.418533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 384/758] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:19.419042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a3549-7eea-4dc9-ac7c-a3e815f5e5c8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:19.459286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25539692-01b0-4f2c-a5a2-616beba0e873 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:19.495901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12d38f-e0cd-44d1-889b-484648359ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:19.496556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12d38f-e0cd-44d1-889b-484648359ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.496834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12d38f-e0cd-44d1-889b-484648359ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.497109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12d38f-e0cd-44d1-889b-484648359ff2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.505925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12d38f-e0cd-44d1-889b-484648359ff2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:19.507749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12d38f-e0cd-44d1-889b-484648359ff2 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-7a12d38f-e0cd-44d1-889b-484648359ff2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:19.519893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:19.520802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:19.521705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:19.521705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:19.521705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:19.521705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:19.521705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:19.521705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:19.521705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:19.521705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:19.521705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 375/759] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:19.522735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606dfe2-70a2-4360-8921-400e8625f4fa None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.522735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606dfe2-70a2-4360-8921-400e8625f4fa None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.522874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606dfe2-70a2-4360-8921-400e8625f4fa None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.524163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 376/760] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:19.524716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606dfe2-70a2-4360-8921-400e8625f4fa None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:19.542088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a3549-7eea-4dc9-ac7c-a3e815f5e5c8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:19.578597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76b334-838b-4ca0-a6ef-7fb1bb090f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:19.579153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76b334-838b-4ca0-a6ef-7fb1bb090f8c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.579429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76b334-838b-4ca0-a6ef-7fb1bb090f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.579734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76b334-838b-4ca0-a6ef-7fb1bb090f8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.586668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76b334-838b-4ca0-a6ef-7fb1bb090f8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:19.588225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76b334-838b-4ca0-a6ef-7fb1bb090f8c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-ed76b334-838b-4ca0-a6ef-7fb1bb090f8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:19.599820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:19.600710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:19.601628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:19.601628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:19.601628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:19.601628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:19.601628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:19.601628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:19.601628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:19.601628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:19.601628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 385/761] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:19.602128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76b334-838b-4ca0-a6ef-7fb1bb090f8c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:19.640414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606dfe2-70a2-4360-8921-400e8625f4fa None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:19.657576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:19.663327 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:19.677693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24dd65-75b2-4b3a-8f2b-28c46c82aeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:19.678329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24dd65-75b2-4b3a-8f2b-28c46c82aeeb None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.678611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24dd65-75b2-4b3a-8f2b-28c46c82aeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.678891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24dd65-75b2-4b3a-8f2b-28c46c82aeeb None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.679610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24dd65-75b2-4b3a-8f2b-28c46c82aeeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:19.679914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24dd65-75b2-4b3a-8f2b-28c46c82aeeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:19.681437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24dd65-75b2-4b3a-8f2b-28c46c82aeeb None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.694768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 377/762] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:19.695986 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f937c6-14ec-4acd-be7c-322cb2a9c7e2 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.696269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f937c6-14ec-4acd-be7c-322cb2a9c7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.697340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f937c6-14ec-4acd-be7c-322cb2a9c7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.713857 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c4f937c6-14ec-4acd-be7c-322cb2a9c7e2 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:19.715392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c4f937c6-14ec-4acd-be7c-322cb2a9c7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:19.716875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76b334-838b-4ca0-a6ef-7fb1bb090f8c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:19.753568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9863e93-3503-4a53-96ba-d72a4d6c8d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:19.754178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9863e93-3503-4a53-96ba-d72a4d6c8d34 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.754467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9863e93-3503-4a53-96ba-d72a4d6c8d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.754727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9863e93-3503-4a53-96ba-d72a4d6c8d34 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.755420 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9863e93-3503-4a53-96ba-d72a4d6c8d34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:19.755755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9863e93-3503-4a53-96ba-d72a4d6c8d34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:19.757201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9863e93-3503-4a53-96ba-d72a4d6c8d34 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.769865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 386/763] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:19.770717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9863e93-3503-4a53-96ba-d72a4d6c8d34 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:19.792735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 378/764] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:19.793547 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f937c6-14ec-4acd-be7c-322cb2a9c7e2 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:19.903933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f937c6-14ec-4acd-be7c-322cb2a9c7e2 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:19.915770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9863e93-3503-4a53-96ba-d72a4d6c8d34 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:19.917351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:19.929652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:19.948920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5d7a5-5b0d-4640-899c-1bdeeeb799d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:19.949581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5d7a5-5b0d-4640-899c-1bdeeeb799d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.949863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5d7a5-5b0d-4640-899c-1bdeeeb799d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.950138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5d7a5-5b0d-4640-899c-1bdeeeb799d3 None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/7956a7748ec94a618eb466f5536220b0/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.952278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5d7a5-5b0d-4640-899c-1bdeeeb799d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=7956a7748ec94a618eb466f5536220b0, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:19.954767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5d7a5-5b0d-4640-899c-1bdeeeb799d3 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.960416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea084090-63ed-4eba-88ad-4072bce0c471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:19.961034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea084090-63ed-4eba-88ad-4072bce0c471 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:19.961318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea084090-63ed-4eba-88ad-4072bce0c471 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:19.961586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea084090-63ed-4eba-88ad-4072bce0c471 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:19.962304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea084090-63ed-4eba-88ad-4072bce0c471 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:19.962606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea084090-63ed-4eba-88ad-4072bce0c471 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:19.964061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea084090-63ed-4eba-88ad-4072bce0c471 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:19.976178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 387/765] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:19.980151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea084090-63ed-4eba-88ad-4072bce0c471 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:20.056462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 379/766] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:10:19 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/7956a7748ec94a618eb466f5536220b0/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:10:20.057764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5d7a5-5b0d-4640-899c-1bdeeeb799d3 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:20.112846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea084090-63ed-4eba-88ad-4072bce0c471 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:20.164216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13adc01-8acf-44c1-a575-d2f31a7f513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:20.165384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13adc01-8acf-44c1-a575-d2f31a7f513d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:20.165665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13adc01-8acf-44c1-a575-d2f31a7f513d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:20.165936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13adc01-8acf-44c1-a575-d2f31a7f513d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:20.166676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13adc01-8acf-44c1-a575-d2f31a7f513d None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=b48aa41ff66b42329926bc42379d9800) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:20.167471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13adc01-8acf-44c1-a575-d2f31a7f513d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:20.169397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13adc01-8acf-44c1-a575-d2f31a7f513d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:20.178213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 388/767] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:19 2022] GET /identity/v3/role_assignments?user.id=b48aa41ff66b42329926bc42379d9800&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=705907b9bdec47cebf88b868a6a56356 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:20.179172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13adc01-8acf-44c1-a575-d2f31a7f513d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:20.212766 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5d7a5-5b0d-4640-899c-1bdeeeb799d3 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:20.227026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:20.265520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e15c7-0e48-4f87-8659-1f15427f1f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:20.266146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e15c7-0e48-4f87-8659-1f15427f1f14 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:20.266424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e15c7-0e48-4f87-8659-1f15427f1f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:20.266697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e15c7-0e48-4f87-8659-1f15427f1f14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:20.267435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e15c7-0e48-4f87-8659-1f15427f1f14 None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=81b72e82b4a34ab6af29ef440d19f6aa) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:20.268478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e15c7-0e48-4f87-8659-1f15427f1f14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:20.269825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e15c7-0e48-4f87-8659-1f15427f1f14 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:20.279691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 380/768] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:20 2022] GET /identity/v3/role_assignments?user.id=81b72e82b4a34ab6af29ef440d19f6aa&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:20.319525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13adc01-8acf-44c1-a575-d2f31a7f513d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:20.358538 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b016f1-0822-4c87-b8b8-c5620760f2a1 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:20.358822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b016f1-0822-4c87-b8b8-c5620760f2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:20.359069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b016f1-0822-4c87-b8b8-c5620760f2a1 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:20.361536 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 381/769] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:10:20.363643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c07908-7b92-4f3e-9249-37fab03835f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:20.364536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c07908-7b92-4f3e-9249-37fab03835f1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:20.365000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c07908-7b92-4f3e-9249-37fab03835f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:20.365477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c07908-7b92-4f3e-9249-37fab03835f1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:20.366263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c07908-7b92-4f3e-9249-37fab03835f1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:20.366669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fae69-ef54-44f0-a232-e92acc8a0781 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:20.367040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fae69-ef54-44f0-a232-e92acc8a0781 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:20.367383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fae69-ef54-44f0-a232-e92acc8a0781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:20.368527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c07908-7b92-4f3e-9249-37fab03835f1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-85c07908-7b92-4f3e-9249-37fab03835f1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:20.382445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:20.383500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:10:20.384396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:20.385139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 389/770] 10.222.0.62 () {56 vars in 1129 bytes} [Mon Sep 5 12:10:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:20.391298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c07908-7b92-4f3e-9249-37fab03835f1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:20.392583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-833fae69-ef54-44f0-a232-e92acc8a0781 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:20.393802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-833fae69-ef54-44f0-a232-e92acc8a0781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:20.450182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:20.535373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c07908-7b92-4f3e-9249-37fab03835f1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:20.539923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:20.542962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 382/771] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:20.585732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ef77c-d13a-4c76-8d08-6cc007906da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:20.586705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ef77c-d13a-4c76-8d08-6cc007906da3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:20.587185 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ef77c-d13a-4c76-8d08-6cc007906da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:20.587697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ef77c-d13a-4c76-8d08-6cc007906da3 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:20.588559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ef77c-d13a-4c76-8d08-6cc007906da3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:20.589060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ef77c-d13a-4c76-8d08-6cc007906da3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:20.592499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ef77c-d13a-4c76-8d08-6cc007906da3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:20.597676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 390/772] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Sep 5 12:10:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:20.603557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fae69-ef54-44f0-a232-e92acc8a0781 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:20.737530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fae69-ef54-44f0-a232-e92acc8a0781 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:20.774304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34b4e4-7ff9-4028-b144-e76dd9249a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:20.774914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34b4e4-7ff9-4028-b144-e76dd9249a18 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:20.775196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34b4e4-7ff9-4028-b144-e76dd9249a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:20.775470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34b4e4-7ff9-4028-b144-e76dd9249a18 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:20.776159 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34b4e4-7ff9-4028-b144-e76dd9249a18 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:20.776465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34b4e4-7ff9-4028-b144-e76dd9249a18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:20.777847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34b4e4-7ff9-4028-b144-e76dd9249a18 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:20.783703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 383/773] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Sep 5 12:10:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:20.789451 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ef77c-d13a-4c76-8d08-6cc007906da3 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:20.834040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34b4e4-7ff9-4028-b144-e76dd9249a18 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:20.876882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:20.944186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ef77c-d13a-4c76-8d08-6cc007906da3 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:21.001881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34b4e4-7ff9-4028-b144-e76dd9249a18 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:21.011640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e50655-3771-41b7-b2f3-903b9852f4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:21.012253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e50655-3771-41b7-b2f3-903b9852f4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:21.012525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e50655-3771-41b7-b2f3-903b9852f4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:21.012789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e50655-3771-41b7-b2f3-903b9852f4b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:21.013484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e50655-3771-41b7-b2f3-903b9852f4b3 None admin] RBAC: Adding query filter params (interface=public, service_id=d77d09d73fe146beaba5a56fd63fbd73, region_id=RegionOne) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:21.013780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e50655-3771-41b7-b2f3-903b9852f4b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:21.015695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e50655-3771-41b7-b2f3-903b9852f4b3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:21.028069 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 391/774] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Sep 5 12:10:20 2022] GET /identity/v3/endpoints?service_id=d77d09d73fe146beaba5a56fd63fbd73&interface=public®ion_id=RegionOne => generated 194 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:21.032132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e50655-3771-41b7-b2f3-903b9852f4b3 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:21.075968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a445f-ad41-4a5e-9a6b-ca55859f8a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:21.076599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a445f-ad41-4a5e-9a6b-ca55859f8a0e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:21.076935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a445f-ad41-4a5e-9a6b-ca55859f8a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:21.077206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a445f-ad41-4a5e-9a6b-ca55859f8a0e None admin] PATH_INFO: `/v3/services/network` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:21.078451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a445f-ad41-4a5e-9a6b-ca55859f8a0e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:21.079847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a445f-ad41-4a5e-9a6b-ca55859f8a0e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:21.091897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-422a445f-ad41-4a5e-9a6b-ca55859f8a0e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:21.095793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:21.096762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:10:21.097612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:21.098353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 384/775] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:20 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:21.104074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a445f-ad41-4a5e-9a6b-ca55859f8a0e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:21.216461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e50655-3771-41b7-b2f3-903b9852f4b3 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:21.268040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:21.281543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a445f-ad41-4a5e-9a6b-ca55859f8a0e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:21.283489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b167d62-fe55-4637-aa61-c64c6eebdbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:21.284116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b167d62-fe55-4637-aa61-c64c6eebdbd7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:21.289858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b167d62-fe55-4637-aa61-c64c6eebdbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:21.289858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b167d62-fe55-4637-aa61-c64c6eebdbd7 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:21.289858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b167d62-fe55-4637-aa61-c64c6eebdbd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:21.289858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b167d62-fe55-4637-aa61-c64c6eebdbd7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:21.297732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 392/776] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Sep 5 12:10:21 2022] GET /identity/v3/services => generated 772 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:21.326051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85042979-e48c-41f4-a01e-5ac90b563517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:21.326659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042979-e48c-41f4-a01e-5ac90b563517 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:21.326934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042979-e48c-41f4-a01e-5ac90b563517 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:21.327805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042979-e48c-41f4-a01e-5ac90b563517 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:21.328505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85042979-e48c-41f4-a01e-5ac90b563517 None admin] RBAC: Adding query filter params (name=network) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:21.328801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85042979-e48c-41f4-a01e-5ac90b563517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:21.330250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85042979-e48c-41f4-a01e-5ac90b563517 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:21.335502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 385/777] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:21.341655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b167d62-fe55-4637-aa61-c64c6eebdbd7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:21.478521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:21.488323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b167d62-fe55-4637-aa61-c64c6eebdbd7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:21.644848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f349d-119e-43c5-a84c-72b52117dc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:21.645910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f349d-119e-43c5-a84c-72b52117dc34 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:21.646474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f349d-119e-43c5-a84c-72b52117dc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:21.646962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f349d-119e-43c5-a84c-72b52117dc34 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:21.647889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f349d-119e-43c5-a84c-72b52117dc34 None admin] RBAC: Adding query filter params (type=network) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:21.648442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f349d-119e-43c5-a84c-72b52117dc34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:21.650120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f349d-119e-43c5-a84c-72b52117dc34 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:21.656969 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 393/778] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:21.662375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85042979-e48c-41f4-a01e-5ac90b563517 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:21.826490 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85042979-e48c-41f4-a01e-5ac90b563517 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:21.871049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92b845-e246-4d27-a471-88271a98b4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:21.871694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92b845-e246-4d27-a471-88271a98b4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:21.871977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92b845-e246-4d27-a471-88271a98b4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:21.872250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92b845-e246-4d27-a471-88271a98b4f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:21.872959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92b845-e246-4d27-a471-88271a98b4f7 None admin] RBAC: Adding query filter params (interface=public, service_id=e85423dc5fc64c62847d077fcc8c7ecf, region_id=RegionOne) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:21.873265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92b845-e246-4d27-a471-88271a98b4f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:21.875174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92b845-e246-4d27-a471-88271a98b4f7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:21.883148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 386/779] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Sep 5 12:10:21 2022] GET /identity/v3/endpoints?service_id=e85423dc5fc64c62847d077fcc8c7ecf&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:21.888270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f349d-119e-43c5-a84c-72b52117dc34 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:22.034437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f349d-119e-43c5-a84c-72b52117dc34 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:22.081557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afc4ef-90c5-463e-afba-b2c1a5cd7174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:22.082166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afc4ef-90c5-463e-afba-b2c1a5cd7174 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:22.082437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afc4ef-90c5-463e-afba-b2c1a5cd7174 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:22.082703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afc4ef-90c5-463e-afba-b2c1a5cd7174 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:22.083305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afc4ef-90c5-463e-afba-b2c1a5cd7174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:22.084676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afc4ef-90c5-463e-afba-b2c1a5cd7174 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:22.090362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 394/780] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Sep 5 12:10:21 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:22.388803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d01af-d8ce-49b8-bd21-9135e3ba48dd None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:22.389312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d01af-d8ce-49b8-bd21-9135e3ba48dd None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:22.389312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d01af-d8ce-49b8-bd21-9135e3ba48dd None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:22.391445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 387/781] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:22.397895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652e850-ac60-4f8e-bac0-2163bffd8288 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:22.398176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652e850-ac60-4f8e-bac0-2163bffd8288 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:22.398446 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652e850-ac60-4f8e-bac0-2163bffd8288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:22.421867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b652e850-ac60-4f8e-bac0-2163bffd8288 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:22.423971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b652e850-ac60-4f8e-bac0-2163bffd8288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:22.442557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:22.513797 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 395/782] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:22.575091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150e494-70fc-41f5-b22a-a86f17317e2f None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:22.575421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150e494-70fc-41f5-b22a-a86f17317e2f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:22.575673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150e494-70fc-41f5-b22a-a86f17317e2f None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:22.577256 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 388/783] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:22.582457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67171ba-df82-4c7c-aca1-003d6981f750 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:22.582741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67171ba-df82-4c7c-aca1-003d6981f750 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:22.583013 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67171ba-df82-4c7c-aca1-003d6981f750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:22.602120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a67171ba-df82-4c7c-aca1-003d6981f750 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:22.603293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a67171ba-df82-4c7c-aca1-003d6981f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:22.672924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48b5ff-81f5-487f-a596-a67b7a762152 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:22.673109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48b5ff-81f5-487f-a596-a67b7a762152 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:22.673371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48b5ff-81f5-487f-a596-a67b7a762152 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:22.674950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 389/784] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:22.679559 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35deb8d1-1494-4382-9ae9-4fd631f86545 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:22.679840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35deb8d1-1494-4382-9ae9-4fd631f86545 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:22.680112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35deb8d1-1494-4382-9ae9-4fd631f86545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:22.696392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 396/785] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:22.704844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-35deb8d1-1494-4382-9ae9-4fd631f86545 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:22.706046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-35deb8d1-1494-4382-9ae9-4fd631f86545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:22.722033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:22.777423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 390/786] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:22.894629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67171ba-df82-4c7c-aca1-003d6981f750 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:22.976487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:23.003834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35deb8d1-1494-4382-9ae9-4fd631f86545 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:23.009209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67171ba-df82-4c7c-aca1-003d6981f750 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:23.050359 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7c037-d3ac-4a92-8a4b-51b2a3f0bcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:23.052268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7c037-d3ac-4a92-8a4b-51b2a3f0bcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.052774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7c037-d3ac-4a92-8a4b-51b2a3f0bcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.053238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7c037-d3ac-4a92-8a4b-51b2a3f0bcc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.070871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7c037-d3ac-4a92-8a4b-51b2a3f0bcc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-45c7c037-d3ac-4a92-8a4b-51b2a3f0bcc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:23.076262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:23.077199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:23.078662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:23.086474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 397/787] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:23.088411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7c037-d3ac-4a92-8a4b-51b2a3f0bcc9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:23.210583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35deb8d1-1494-4382-9ae9-4fd631f86545 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:23.219143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7c037-d3ac-4a92-8a4b-51b2a3f0bcc9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:23.255204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb3557-073b-418c-ade2-b3c59705b711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:23.255879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb3557-073b-418c-ade2-b3c59705b711 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.256161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb3557-073b-418c-ade2-b3c59705b711 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.256435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb3557-073b-418c-ade2-b3c59705b711 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.257098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb3557-073b-418c-ade2-b3c59705b711 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:23.257404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb3557-073b-418c-ade2-b3c59705b711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:23.258701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c18d9-a349-4148-b567-206d4c4d7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:23.258912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb3557-073b-418c-ade2-b3c59705b711 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:23.259744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c18d9-a349-4148-b567-206d4c4d7c81 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.260467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c18d9-a349-4148-b567-206d4c4d7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.261358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c18d9-a349-4148-b567-206d4c4d7c81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.265790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:23.273680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 398/788] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:23.273680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb3557-073b-418c-ade2-b3c59705b711 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:23.286807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c18d9-a349-4148-b567-206d4c4d7c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-138c18d9-a349-4148-b567-206d4c4d7c81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:23.293558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:23.294700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:23.295773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:23.299797 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 391/789] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:23.301018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c18d9-a349-4148-b567-206d4c4d7c81 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:23.316653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:23.382060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb3557-073b-418c-ade2-b3c59705b711 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:23.410776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c18d9-a349-4148-b567-206d4c4d7c81 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:23.419266 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc53e4-1f68-46ea-b58f-c16ab561bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:23.420367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc53e4-1f68-46ea-b58f-c16ab561bfca None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.420541 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc53e4-1f68-46ea-b58f-c16ab561bfca None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.420909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc53e4-1f68-46ea-b58f-c16ab561bfca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.432825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc53e4-1f68-46ea-b58f-c16ab561bfca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e0dc53e4-1f68-46ea-b58f-c16ab561bfca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:23.438335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:23.439562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:23.440493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:23.445807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 399/790] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:23.446770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc53e4-1f68-46ea-b58f-c16ab561bfca None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:23.454255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e5f41-3a5d-42a5-9d46-6fcf9a234876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:23.455124 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e5f41-3a5d-42a5-9d46-6fcf9a234876 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.455602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e5f41-3a5d-42a5-9d46-6fcf9a234876 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.456048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e5f41-3a5d-42a5-9d46-6fcf9a234876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.467251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e5f41-3a5d-42a5-9d46-6fcf9a234876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:23.469505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e5f41-3a5d-42a5-9d46-6fcf9a234876 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-167e5f41-3a5d-42a5-9d46-6fcf9a234876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:23.484844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:23.485876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:23.486782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:23.486782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:23.486782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:23.488941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 392/791] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:23.489909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e5f41-3a5d-42a5-9d46-6fcf9a234876 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:23.517982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:23.555697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:23.562480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc53e4-1f68-46ea-b58f-c16ab561bfca None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:23.598532 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e5f41-3a5d-42a5-9d46-6fcf9a234876 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:23.599630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78641b5e-c988-429f-9534-84ffc72fa9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:23.600249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641b5e-c988-429f-9534-84ffc72fa9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.600527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641b5e-c988-429f-9534-84ffc72fa9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.600800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641b5e-c988-429f-9534-84ffc72fa9f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.601540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78641b5e-c988-429f-9534-84ffc72fa9f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:23.601842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78641b5e-c988-429f-9534-84ffc72fa9f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:23.603383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78641b5e-c988-429f-9534-84ffc72fa9f0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:23.613590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 400/792] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:23.614470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78641b5e-c988-429f-9534-84ffc72fa9f0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:23.639717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48baba8-634a-4648-a847-b304a5e63219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:23.640712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48baba8-634a-4648-a847-b304a5e63219 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.641265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48baba8-634a-4648-a847-b304a5e63219 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.641731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48baba8-634a-4648-a847-b304a5e63219 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.642575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48baba8-634a-4648-a847-b304a5e63219 None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:23.643068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48baba8-634a-4648-a847-b304a5e63219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:23.644672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48baba8-634a-4648-a847-b304a5e63219 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:23.653574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 393/793] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:23.654612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48baba8-634a-4648-a847-b304a5e63219 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:23.729032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78641b5e-c988-429f-9534-84ffc72fa9f0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:23.752096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:23.760408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48baba8-634a-4648-a847-b304a5e63219 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:23.763697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b44f68-7a5a-46d9-bac7-cb0a21f6864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:23.766466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b44f68-7a5a-46d9-bac7-cb0a21f6864b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.766466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b44f68-7a5a-46d9-bac7-cb0a21f6864b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.766466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b44f68-7a5a-46d9-bac7-cb0a21f6864b None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.766466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b44f68-7a5a-46d9-bac7-cb0a21f6864b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:23.766466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b44f68-7a5a-46d9-bac7-cb0a21f6864b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:23.767750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b44f68-7a5a-46d9-bac7-cb0a21f6864b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:23.779636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 401/794] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:23.780222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b44f68-7a5a-46d9-bac7-cb0a21f6864b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:23.791679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:23.803348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbb435-7a45-4c47-84df-b233fc513c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:23.804017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbb435-7a45-4c47-84df-b233fc513c7f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.804414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbb435-7a45-4c47-84df-b233fc513c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.804698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbb435-7a45-4c47-84df-b233fc513c7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.895527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b44f68-7a5a-46d9-bac7-cb0a21f6864b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:23.930056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbb435-7a45-4c47-84df-b233fc513c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:23.931508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbb435-7a45-4c47-84df-b233fc513c7f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:23.933360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe6c9f-0be8-4a16-9c54-6c051b377c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:23.933989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe6c9f-0be8-4a16-9c54-6c051b377c88 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.934273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe6c9f-0be8-4a16-9c54-6c051b377c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.934543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe6c9f-0be8-4a16-9c54-6c051b377c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.944810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe6c9f-0be8-4a16-9c54-6c051b377c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:23.946256 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe6c9f-0be8-4a16-9c54-6c051b377c88 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-0fcbb435-7a45-4c47-84df-b233fc513c7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:23.947180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:23.948153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:23.949034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:23.949034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:23.949034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:23.949034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 394/795] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:23.949855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b4d84-a21a-4a34-9802-9cf88eff6d15 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:23.949855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b4d84-a21a-4a34-9802-9cf88eff6d15 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:23.950014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b4d84-a21a-4a34-9802-9cf88eff6d15 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:23.951365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 395/796] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:23.956082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b4d84-a21a-4a34-9802-9cf88eff6d15 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-38fe6c9f-0be8-4a16-9c54-6c051b377c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:23.969668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:23.970803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:23.972022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:23.972022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:23.972022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:23.972022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 402/797] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:23.972022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe6c9f-0be8-4a16-9c54-6c051b377c88 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.008817 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:24.080318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b4d84-a21a-4a34-9802-9cf88eff6d15 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:24.086383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe6c9f-0be8-4a16-9c54-6c051b377c88 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:24.118658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b574a-58aa-4de8-9bc6-6e3940bf61c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:24.119636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b574a-58aa-4de8-9bc6-6e3940bf61c0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.120061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b574a-58aa-4de8-9bc6-6e3940bf61c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.120493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b574a-58aa-4de8-9bc6-6e3940bf61c0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.123592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffc12a-ac0c-44da-8eed-b6e917069652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:24.124196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc12a-ac0c-44da-8eed-b6e917069652 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.124479 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc12a-ac0c-44da-8eed-b6e917069652 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.124726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc12a-ac0c-44da-8eed-b6e917069652 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.125476 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffc12a-ac0c-44da-8eed-b6e917069652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:24.125776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffc12a-ac0c-44da-8eed-b6e917069652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:24.127118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffc12a-ac0c-44da-8eed-b6e917069652 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:24.129346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b574a-58aa-4de8-9bc6-6e3940bf61c0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:24.130769 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b574a-58aa-4de8-9bc6-6e3940bf61c0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:24.138356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 403/798] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:24.139725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1158ff-5480-43f1-a24c-a09c0be03f52 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.139950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1158ff-5480-43f1-a24c-a09c0be03f52 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.140250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1158ff-5480-43f1-a24c-a09c0be03f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-f08b574a-58aa-4de8-9bc6-6e3940bf61c0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:24.147581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:24.153271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 396/799] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:24.154531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b574a-58aa-4de8-9bc6-6e3940bf61c0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.166556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6d1158ff-5480-43f1-a24c-a09c0be03f52 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:24.167765 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6d1158ff-5480-43f1-a24c-a09c0be03f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:24.190296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:24.238299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 404/800] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:24.239408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1158ff-5480-43f1-a24c-a09c0be03f52 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.261995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b574a-58aa-4de8-9bc6-6e3940bf61c0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:24.266247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:24.300178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a96d97-9768-4b31-b8e0-9fe694345d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:24.300846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a96d97-9768-4b31-b8e0-9fe694345d01 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.301118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a96d97-9768-4b31-b8e0-9fe694345d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.301398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a96d97-9768-4b31-b8e0-9fe694345d01 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.302096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a96d97-9768-4b31-b8e0-9fe694345d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:24.302406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a96d97-9768-4b31-b8e0-9fe694345d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:24.303942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a96d97-9768-4b31-b8e0-9fe694345d01 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:24.320463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 397/801] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:24.321480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a96d97-9768-4b31-b8e0-9fe694345d01 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.354122 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1158ff-5480-43f1-a24c-a09c0be03f52 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:24.391865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1759a-5106-4ba9-9dc0-759260a79724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:24.392821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1759a-5106-4ba9-9dc0-759260a79724 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.393305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1759a-5106-4ba9-9dc0-759260a79724 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.393896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1759a-5106-4ba9-9dc0-759260a79724 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.401651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1759a-5106-4ba9-9dc0-759260a79724 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:24.403488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1759a-5106-4ba9-9dc0-759260a79724 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-cfd1759a-5106-4ba9-9dc0-759260a79724 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:24.417311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:24.418340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:24.419302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:24.419302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:24.419302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:24.419302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:24.419302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:24.419302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:24.419302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:24.419302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:24.419302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:24.419302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:24.421611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:24.427745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 405/802] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Sep 5 12:10:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:24.428328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a96d97-9768-4b31-b8e0-9fe694345d01 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:24.429025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1759a-5106-4ba9-9dc0-759260a79724 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.465966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88568f26-1e80-4779-ba9b-39688fa02e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:24.466576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88568f26-1e80-4779-ba9b-39688fa02e48 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.466851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88568f26-1e80-4779-ba9b-39688fa02e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.467121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88568f26-1e80-4779-ba9b-39688fa02e48 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.467893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88568f26-1e80-4779-ba9b-39688fa02e48 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:24.468250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88568f26-1e80-4779-ba9b-39688fa02e48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:24.469669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88568f26-1e80-4779-ba9b-39688fa02e48 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:24.487168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 398/803] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Sep 5 12:10:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:24.487990 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88568f26-1e80-4779-ba9b-39688fa02e48 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.525194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:24.540519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1759a-5106-4ba9-9dc0-759260a79724 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:24.574115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3499b93-5e32-4e1a-9d6d-ab45374bdffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:24.574668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3499b93-5e32-4e1a-9d6d-ab45374bdffc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.574945 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3499b93-5e32-4e1a-9d6d-ab45374bdffc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.575221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3499b93-5e32-4e1a-9d6d-ab45374bdffc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.582575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3499b93-5e32-4e1a-9d6d-ab45374bdffc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:24.584027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3499b93-5e32-4e1a-9d6d-ab45374bdffc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:24.589984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88568f26-1e80-4779-ba9b-39688fa02e48 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-b3499b93-5e32-4e1a-9d6d-ab45374bdffc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:24.595979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:24.601638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 406/804] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Sep 5 12:10:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde597fe-fb5a-4004-88db-01a6d6936c66 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.602908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde597fe-fb5a-4004-88db-01a6d6936c66 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.603889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde597fe-fb5a-4004-88db-01a6d6936c66 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.603889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 407/805] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:24.603889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde597fe-fb5a-4004-88db-01a6d6936c66 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.632449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea934cd-c0ff-412e-afac-afbcdc6000da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:24.633045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea934cd-c0ff-412e-afac-afbcdc6000da None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.633319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea934cd-c0ff-412e-afac-afbcdc6000da None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.633587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea934cd-c0ff-412e-afac-afbcdc6000da None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.634296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea934cd-c0ff-412e-afac-afbcdc6000da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:24.634590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea934cd-c0ff-412e-afac-afbcdc6000da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:24.636040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea934cd-c0ff-412e-afac-afbcdc6000da None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:24.652593 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 399/806] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Sep 5 12:10:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:24.653456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea934cd-c0ff-412e-afac-afbcdc6000da None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.667428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:24.701223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde597fe-fb5a-4004-88db-01a6d6936c66 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:24.735199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea8ad9-08bf-4d32-977d-36052e2c7944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:24.735857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea8ad9-08bf-4d32-977d-36052e2c7944 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.736136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea8ad9-08bf-4d32-977d-36052e2c7944 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.736415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea8ad9-08bf-4d32-977d-36052e2c7944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.747209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea8ad9-08bf-4d32-977d-36052e2c7944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:24.748676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea8ad9-08bf-4d32-977d-36052e2c7944 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:24.750718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea934cd-c0ff-412e-afac-afbcdc6000da None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:24.751693 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-36ea8ad9-08bf-4d32-977d-36052e2c7944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:24.763454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:24.764488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:24.765982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:24.765982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:24.765982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:24.765982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 408/807] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:24.767205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea8ad9-08bf-4d32-977d-36052e2c7944 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.785523 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac54e33-a162-4cf1-bdb4-35177b6a5b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:24.786145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac54e33-a162-4cf1-bdb4-35177b6a5b99 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.786419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac54e33-a162-4cf1-bdb4-35177b6a5b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.786682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac54e33-a162-4cf1-bdb4-35177b6a5b99 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.787253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac54e33-a162-4cf1-bdb4-35177b6a5b99 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:24.788649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac54e33-a162-4cf1-bdb4-35177b6a5b99 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-cac54e33-a162-4cf1-bdb4-35177b6a5b99 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:24.797506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:24.798597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:24.799449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 400/808] 10.222.0.62 () {56 vars in 1129 bytes} [Mon Sep 5 12:10:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:24.800781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5e06-a06f-4e4e-8bfc-3f933dd55b0c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.800781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5e06-a06f-4e4e-8bfc-3f933dd55b0c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.800781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5e06-a06f-4e4e-8bfc-3f933dd55b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.817744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ee5d5e06-a06f-4e4e-8bfc-3f933dd55b0c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:24.818922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ee5d5e06-a06f-4e4e-8bfc-3f933dd55b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:24.868077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea8ad9-08bf-4d32-977d-36052e2c7944 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:24.883761 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 401/809] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:24.884561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5e06-a06f-4e4e-8bfc-3f933dd55b0c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.902625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:24.906223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4cc05-4226-4fb5-a762-859d0db4b781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:24.906826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4cc05-4226-4fb5-a762-859d0db4b781 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:24.907101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4cc05-4226-4fb5-a762-859d0db4b781 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:24.907367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4cc05-4226-4fb5-a762-859d0db4b781 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:24.908112 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4cc05-4226-4fb5-a762-859d0db4b781 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:24.908407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4cc05-4226-4fb5-a762-859d0db4b781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:24.909804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4cc05-4226-4fb5-a762-859d0db4b781 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:24.926434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 409/810] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Sep 5 12:10:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:24.927222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4cc05-4226-4fb5-a762-859d0db4b781 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:24.982650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:25.005520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5e06-a06f-4e4e-8bfc-3f933dd55b0c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.028755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4cc05-4226-4fb5-a762-859d0db4b781 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.040598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0def88-f858-4d0e-904d-ec729dc1356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.041197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0def88-f858-4d0e-904d-ec729dc1356a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.041470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0def88-f858-4d0e-904d-ec729dc1356a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.041735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0def88-f858-4d0e-904d-ec729dc1356a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.069593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29f7c1-ba51-45cc-9988-d316539b3a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.070320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29f7c1-ba51-45cc-9988-d316539b3a7c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.070595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29f7c1-ba51-45cc-9988-d316539b3a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.070856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29f7c1-ba51-45cc-9988-d316539b3a7c None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.071498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29f7c1-ba51-45cc-9988-d316539b3a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:25.071835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29f7c1-ba51-45cc-9988-d316539b3a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.073187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29f7c1-ba51-45cc-9988-d316539b3a7c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.085793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 410/811] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:25.086589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29f7c1-ba51-45cc-9988-d316539b3a7c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.119016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0def88-f858-4d0e-904d-ec729dc1356a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.120440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0def88-f858-4d0e-904d-ec729dc1356a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.128101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:25.193384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29f7c1-ba51-45cc-9988-d316539b3a7c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.206418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:25.219362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 402/812] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:10:25.220658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0def88-f858-4d0e-904d-ec729dc1356a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.234895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a124e-d940-400b-beda-bd77646c2c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.235630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a124e-d940-400b-beda-bd77646c2c25 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.236274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a124e-d940-400b-beda-bd77646c2c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.236702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a124e-d940-400b-beda-bd77646c2c25 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.237588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a124e-d940-400b-beda-bd77646c2c25 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:25.237892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a124e-d940-400b-beda-bd77646c2c25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.239400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a124e-d940-400b-beda-bd77646c2c25 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.243762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 411/813] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:25.244530 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a124e-d940-400b-beda-bd77646c2c25 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.323548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0def88-f858-4d0e-904d-ec729dc1356a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.331695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:25.341110 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a124e-d940-400b-beda-bd77646c2c25 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.357493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17767127-3283-4c17-81b3-164371415a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.358109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17767127-3283-4c17-81b3-164371415a83 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.358383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17767127-3283-4c17-81b3-164371415a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.358649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17767127-3283-4c17-81b3-164371415a83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.368373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17767127-3283-4c17-81b3-164371415a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.370249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17767127-3283-4c17-81b3-164371415a83 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.373497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79093452-5d8b-4d14-b20f-1cc782929d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.374126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79093452-5d8b-4d14-b20f-1cc782929d3b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.374919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79093452-5d8b-4d14-b20f-1cc782929d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.375191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79093452-5d8b-4d14-b20f-1cc782929d3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.382005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79093452-5d8b-4d14-b20f-1cc782929d3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.383416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79093452-5d8b-4d14-b20f-1cc782929d3b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-17767127-3283-4c17-81b3-164371415a83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:25.385398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:25.386345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:25.387238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:25.387238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:25.387238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:25.387238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 403/814] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:25.387238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17767127-3283-4c17-81b3-164371415a83 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-79093452-5d8b-4d14-b20f-1cc782929d3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:25.395324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:25.396261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:25.397151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:25.397151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:25.397151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:25.397151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:25.397151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:25.397151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:25.397151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:25.397151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:25.397151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 412/815] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:25.398348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79093452-5d8b-4d14-b20f-1cc782929d3b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.436481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:25.496048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17767127-3283-4c17-81b3-164371415a83 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.496608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79093452-5d8b-4d14-b20f-1cc782929d3b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.529791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70100118-1139-495f-bfcd-361fff3ade75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.530402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70100118-1139-495f-bfcd-361fff3ade75 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.530604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ed41f-a25d-40d6-a56c-65eda844d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.530720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70100118-1139-495f-bfcd-361fff3ade75 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.530927 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70100118-1139-495f-bfcd-361fff3ade75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.531218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed41f-a25d-40d6-a56c-65eda844d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.531492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed41f-a25d-40d6-a56c-65eda844d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.531788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed41f-a25d-40d6-a56c-65eda844d8b0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.532361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ed41f-a25d-40d6-a56c-65eda844d8b0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.533723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ed41f-a25d-40d6-a56c-65eda844d8b0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.541065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70100118-1139-495f-bfcd-361fff3ade75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-303ed41f-a25d-40d6-a56c-65eda844d8b0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:25.542374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:25.543401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 12:10:25.544294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:25.545032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70100118-1139-495f-bfcd-361fff3ade75 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.545104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 404/816] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/services/network => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:25.545104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ed41f-a25d-40d6-a56c-65eda844d8b0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-70100118-1139-495f-bfcd-361fff3ade75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:25.557960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:25.559021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:25.560039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:25.560039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:25.560039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:25.560039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 413/817] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:25.560039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70100118-1139-495f-bfcd-361fff3ade75 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.570693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:25.645997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ed41f-a25d-40d6-a56c-65eda844d8b0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.652087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:25.661540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70100118-1139-495f-bfcd-361fff3ade75 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.678295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f66c8-3151-4bf7-855b-d872f61743d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.678887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f66c8-3151-4bf7-855b-d872f61743d3 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.679163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f66c8-3151-4bf7-855b-d872f61743d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.679431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f66c8-3151-4bf7-855b-d872f61743d3 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.680085 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f66c8-3151-4bf7-855b-d872f61743d3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:25.680421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f66c8-3151-4bf7-855b-d872f61743d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.681772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f66c8-3151-4bf7-855b-d872f61743d3 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.686427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 405/818] 10.222.0.62 () {56 vars in 1124 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:25.687714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f66c8-3151-4bf7-855b-d872f61743d3 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.695582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675897d0-ec08-4316-892f-dfdadd0658db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.696192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675897d0-ec08-4316-892f-dfdadd0658db None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.696467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675897d0-ec08-4316-892f-dfdadd0658db None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.696736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675897d0-ec08-4316-892f-dfdadd0658db None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.697379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675897d0-ec08-4316-892f-dfdadd0658db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:25.697677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675897d0-ec08-4316-892f-dfdadd0658db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.699048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675897d0-ec08-4316-892f-dfdadd0658db None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.711163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 414/819] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:25.711937 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675897d0-ec08-4316-892f-dfdadd0658db None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.772674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:25.786651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f66c8-3151-4bf7-855b-d872f61743d3 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.809574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675897d0-ec08-4316-892f-dfdadd0658db None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.821148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec001a-d3fd-4eeb-abcb-0af79a0fa618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.821749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec001a-d3fd-4eeb-abcb-0af79a0fa618 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.822030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec001a-d3fd-4eeb-abcb-0af79a0fa618 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.822301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec001a-d3fd-4eeb-abcb-0af79a0fa618 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.822988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec001a-d3fd-4eeb-abcb-0af79a0fa618 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:25.823291 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec001a-d3fd-4eeb-abcb-0af79a0fa618 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.824749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec001a-d3fd-4eeb-abcb-0af79a0fa618 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.835873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 406/820] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:25.836636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec001a-d3fd-4eeb-abcb-0af79a0fa618 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.847181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bfbdd-9958-429b-b858-2fe991e5b7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.847807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bfbdd-9958-429b-b858-2fe991e5b7fa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.848084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bfbdd-9958-429b-b858-2fe991e5b7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.848351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bfbdd-9958-429b-b858-2fe991e5b7fa None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.848970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bfbdd-9958-429b-b858-2fe991e5b7fa None admin] RBAC: Adding query filter params (name=network) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:25.849267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bfbdd-9958-429b-b858-2fe991e5b7fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.850599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bfbdd-9958-429b-b858-2fe991e5b7fa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.855077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 415/821] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:25.856040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bfbdd-9958-429b-b858-2fe991e5b7fa None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.872260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:25.936015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec001a-d3fd-4eeb-abcb-0af79a0fa618 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.954502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bfbdd-9958-429b-b858-2fe991e5b7fa None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:25.969014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486f745-fd36-4413-a581-d1f95335014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.969607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486f745-fd36-4413-a581-d1f95335014c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.969884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486f745-fd36-4413-a581-d1f95335014c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.970153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486f745-fd36-4413-a581-d1f95335014c None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.970781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486f745-fd36-4413-a581-d1f95335014c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:25.971082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486f745-fd36-4413-a581-d1f95335014c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.972463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486f745-fd36-4413-a581-d1f95335014c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.989102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 407/822] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:25.989102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486f745-fd36-4413-a581-d1f95335014c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:25.989331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080bbba-f04b-4d28-8905-e4e33f57cc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:25.989331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080bbba-f04b-4d28-8905-e4e33f57cc95 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:25.989331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080bbba-f04b-4d28-8905-e4e33f57cc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:25.989331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080bbba-f04b-4d28-8905-e4e33f57cc95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:25.989633 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080bbba-f04b-4d28-8905-e4e33f57cc95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:25.990609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080bbba-f04b-4d28-8905-e4e33f57cc95 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:25.995706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:26.084378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486f745-fd36-4413-a581-d1f95335014c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:26.105452 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 416/823] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:10:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:26.106323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080bbba-f04b-4d28-8905-e4e33f57cc95 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:26.126439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d53336-c89d-491e-bd06-4138c59fdbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:26.127084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53336-c89d-491e-bd06-4138c59fdbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:26.127363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53336-c89d-491e-bd06-4138c59fdbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:26.127663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53336-c89d-491e-bd06-4138c59fdbe5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:26.135206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d53336-c89d-491e-bd06-4138c59fdbe5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:26.136725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d53336-c89d-491e-bd06-4138c59fdbe5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-90d53336-c89d-491e-bd06-4138c59fdbe5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:26.149435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:26.150445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:26.152021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:26.152021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:26.152021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:26.152021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:26.152021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:26.152021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:26.152021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:26.152021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:26.152021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 408/824] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:26.152021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d53336-c89d-491e-bd06-4138c59fdbe5 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:26.210561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:26.232130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:26.243401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080bbba-f04b-4d28-8905-e4e33f57cc95 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:26.278939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d53336-c89d-491e-bd06-4138c59fdbe5 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:26.286755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca4def-cd14-4e68-921e-ba67a1d8df11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:26.287529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca4def-cd14-4e68-921e-ba67a1d8df11 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:26.287888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca4def-cd14-4e68-921e-ba67a1d8df11 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:26.288175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca4def-cd14-4e68-921e-ba67a1d8df11 None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/b48aa41ff66b42329926bc42379d9800/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:26.290449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca4def-cd14-4e68-921e-ba67a1d8df11 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=b48aa41ff66b42329926bc42379d9800, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:26.291993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca4def-cd14-4e68-921e-ba67a1d8df11 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:26.316716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edea6d-b913-47dd-8ca2-f4fc4225b08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:26.317352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edea6d-b913-47dd-8ca2-f4fc4225b08d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:26.317634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edea6d-b913-47dd-8ca2-f4fc4225b08d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:26.317904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edea6d-b913-47dd-8ca2-f4fc4225b08d None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:26.318529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edea6d-b913-47dd-8ca2-f4fc4225b08d None admin] RBAC: Adding query filter params (type=network) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:26.318829 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edea6d-b913-47dd-8ca2-f4fc4225b08d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:26.320224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edea6d-b913-47dd-8ca2-f4fc4225b08d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:26.325113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 409/825] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:26.325871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edea6d-b913-47dd-8ca2-f4fc4225b08d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:26.351354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 417/826] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:10:26 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/b48aa41ff66b42329926bc42379d9800/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:10:26.359702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca4def-cd14-4e68-921e-ba67a1d8df11 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:26.476436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edea6d-b913-47dd-8ca2-f4fc4225b08d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:26.477590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca4def-cd14-4e68-921e-ba67a1d8df11 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:26.513357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:26.516886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:26.523927 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfab9e-d020-49df-975b-09ae37b5a7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:26.524537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfab9e-d020-49df-975b-09ae37b5a7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:26.524651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8cc82-046b-4220-824a-2d65e2560487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:26.524808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfab9e-d020-49df-975b-09ae37b5a7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:26.525164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfab9e-d020-49df-975b-09ae37b5a7c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:26.525537 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8cc82-046b-4220-824a-2d65e2560487 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:26.525841 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfab9e-d020-49df-975b-09ae37b5a7c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:26.525945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8cc82-046b-4220-824a-2d65e2560487 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:26.526138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfab9e-d020-49df-975b-09ae37b5a7c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:26.526395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8cc82-046b-4220-824a-2d65e2560487 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:26.527752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfab9e-d020-49df-975b-09ae37b5a7c8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:26.537206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8cc82-046b-4220-824a-2d65e2560487 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:26.538160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8cc82-046b-4220-824a-2d65e2560487 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:26.543329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 418/827] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:26.544643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfab9e-d020-49df-975b-09ae37b5a7c8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-14d8cc82-046b-4220-824a-2d65e2560487 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:26.552851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:26.554808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:26.555742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:26.555742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:26.555742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:26.555742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:26.555742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:26.555742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:26.555742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:26.555742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:26.558809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 410/828] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:26.559659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8cc82-046b-4220-824a-2d65e2560487 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:26.657222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfab9e-d020-49df-975b-09ae37b5a7c8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:26.688227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8cc82-046b-4220-824a-2d65e2560487 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:26.693697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd433d-73a6-4a87-b1bf-cb02d369892e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:26.694387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd433d-73a6-4a87-b1bf-cb02d369892e None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:26.694734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd433d-73a6-4a87-b1bf-cb02d369892e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:26.695071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd433d-73a6-4a87-b1bf-cb02d369892e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:26.695854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd433d-73a6-4a87-b1bf-cb02d369892e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:26.697277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd433d-73a6-4a87-b1bf-cb02d369892e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:26.730809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8ef58-da88-44ff-af12-6608618cbb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:26.731562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8ef58-da88-44ff-af12-6608618cbb0b None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:26.731835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8ef58-da88-44ff-af12-6608618cbb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:26.732107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8ef58-da88-44ff-af12-6608618cbb0b None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/81b72e82b4a34ab6af29ef440d19f6aa/roles/dd50550e774349d6af84f593eeeec10d` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:26.747906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8ef58-da88-44ff-af12-6608618cbb0b None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=81b72e82b4a34ab6af29ef440d19f6aa, role_id=dd50550e774349d6af84f593eeeec10d)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:26.750070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8ef58-da88-44ff-af12-6608618cbb0b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:26.776890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 419/829] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:10:26 2022] POST /identity/v3/endpoints => generated 337 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:26.778236 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd433d-73a6-4a87-b1bf-cb02d369892e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:26.830036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 411/830] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:10:26 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/81b72e82b4a34ab6af29ef440d19f6aa/roles/dd50550e774349d6af84f593eeeec10d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:10:26.838482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:26.918424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd433d-73a6-4a87-b1bf-cb02d369892e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:26.962081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c1fce-a1d3-4343-97fd-4fe509d9c43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:26.962898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c1fce-a1d3-4343-97fd-4fe509d9c43f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:26.963288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c1fce-a1d3-4343-97fd-4fe509d9c43f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:26.963709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c1fce-a1d3-4343-97fd-4fe509d9c43f None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:26.964544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c1fce-a1d3-4343-97fd-4fe509d9c43f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:26.965247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c1fce-a1d3-4343-97fd-4fe509d9c43f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:26.966785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c1fce-a1d3-4343-97fd-4fe509d9c43f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:26.981956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 420/831] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:26.988194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8ef58-da88-44ff-af12-6608618cbb0b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:27.031951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:27.105395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8ef58-da88-44ff-af12-6608618cbb0b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:27.147416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1c489-b7f7-4427-9979-1471926acbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:27.148378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c489-b7f7-4427-9979-1471926acbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:27.148859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c489-b7f7-4427-9979-1471926acbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:27.149314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c489-b7f7-4427-9979-1471926acbd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:27.150211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1c489-b7f7-4427-9979-1471926acbd1 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=7956a7748ec94a618eb466f5536220b0) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:27.150691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1c489-b7f7-4427-9979-1471926acbd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:27.152326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1c489-b7f7-4427-9979-1471926acbd1 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:27.163654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 412/832] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:26 2022] GET /identity/v3/role_assignments?user.id=7956a7748ec94a618eb466f5536220b0&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=705907b9bdec47cebf88b868a6a56356 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:28.090660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4b4af-0a1f-4267-ae72-8e3048efeaf3 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:28.091176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4b4af-0a1f-4267-ae72-8e3048efeaf3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:28.091176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4b4af-0a1f-4267-ae72-8e3048efeaf3 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:28.092813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 421/833] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:28.098621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee6ce0-25f9-45e7-8446-68b5d0ca611f None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:28.099018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee6ce0-25f9-45e7-8446-68b5d0ca611f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:28.099186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee6ce0-25f9-45e7-8446-68b5d0ca611f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:28.117154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-26ee6ce0-25f9-45e7-8446-68b5d0ca611f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:28.118346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-26ee6ce0-25f9-45e7-8446-68b5d0ca611f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:28.191769 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 413/834] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:28.426836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6323-275b-4675-8112-217ac78ea78a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:28.427480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6323-275b-4675-8112-217ac78ea78a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:28.427979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6323-275b-4675-8112-217ac78ea78a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:28.429738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 422/835] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:28.435129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dad493-30c4-4749-a207-5b8065a29f53 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:28.435679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dad493-30c4-4749-a207-5b8065a29f53 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:28.436148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dad493-30c4-4749-a207-5b8065a29f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:28.454558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c4dad493-30c4-4749-a207-5b8065a29f53 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:28.456070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c4dad493-30c4-4749-a207-5b8065a29f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:28.466060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6323-275b-4675-8112-217ac78ea78a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:28.482581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:28.532210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 414/836] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:28.580167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6323-275b-4675-8112-217ac78ea78a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:28.581734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:28.617383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c5c86-df88-4487-95e5-50036e9f1dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:28.618020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c5c86-df88-4487-95e5-50036e9f1dee None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:28.618297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c5c86-df88-4487-95e5-50036e9f1dee None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:28.618565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c5c86-df88-4487-95e5-50036e9f1dee None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:28.619206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c5c86-df88-4487-95e5-50036e9f1dee None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:28.620596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c5c86-df88-4487-95e5-50036e9f1dee None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-409c5c86-df88-4487-95e5-50036e9f1dee None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:28.629646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:28.630599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:28.631465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 423/837] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:10:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:28.639218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dad493-30c4-4749-a207-5b8065a29f53 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:28.746480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dad493-30c4-4749-a207-5b8065a29f53 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:28.782487 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71788217-d61d-46c4-967c-c0a48d89baba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:28.783099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71788217-d61d-46c4-967c-c0a48d89baba None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:28.783378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71788217-d61d-46c4-967c-c0a48d89baba None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:28.783678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71788217-d61d-46c4-967c-c0a48d89baba None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:28.784312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71788217-d61d-46c4-967c-c0a48d89baba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:28.784612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71788217-d61d-46c4-967c-c0a48d89baba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:28.785997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71788217-d61d-46c4-967c-c0a48d89baba None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:28.791072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 415/838] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:10:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:28.796734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c5c86-df88-4487-95e5-50036e9f1dee None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:28.823289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71788217-d61d-46c4-967c-c0a48d89baba None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:28.859721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:28.917870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c5c86-df88-4487-95e5-50036e9f1dee None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:28.941746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71788217-d61d-46c4-967c-c0a48d89baba None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:28.956879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b25ad0-3d9a-4750-8471-a4dff7de4f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:28.957488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25ad0-3d9a-4750-8471-a4dff7de4f3c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:28.957768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25ad0-3d9a-4750-8471-a4dff7de4f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:28.958037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25ad0-3d9a-4750-8471-a4dff7de4f3c None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:28.958663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b25ad0-3d9a-4750-8471-a4dff7de4f3c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:28.958965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b25ad0-3d9a-4750-8471-a4dff7de4f3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:28.960355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b25ad0-3d9a-4750-8471-a4dff7de4f3c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:28.964795 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 424/839] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:10:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:28.966201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366cfcb-7e74-4fd9-8cd4-867241df0b79 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:28.966482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366cfcb-7e74-4fd9-8cd4-867241df0b79 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:28.966756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366cfcb-7e74-4fd9-8cd4-867241df0b79 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:28.968125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 425/840] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:28.973723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2588b88-63d2-4167-a1b2-deaaf7885923 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:28.974007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2588b88-63d2-4167-a1b2-deaaf7885923 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:28.974283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2588b88-63d2-4167-a1b2-deaaf7885923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:28.991599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad2f2c-6bb0-4757-b0fa-02e7222cc6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:28.992252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad2f2c-6bb0-4757-b0fa-02e7222cc6ec None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:28.992532 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad2f2c-6bb0-4757-b0fa-02e7222cc6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:28.992806 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad2f2c-6bb0-4757-b0fa-02e7222cc6ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.006428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad2f2c-6bb0-4757-b0fa-02e7222cc6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:29.007080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e2588b88-63d2-4167-a1b2-deaaf7885923 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:29.008556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e2588b88-63d2-4167-a1b2-deaaf7885923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-e8ad2f2c-6bb0-4757-b0fa-02e7222cc6ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:29.011278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:29.012398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:29.013287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 416/841] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:29.022512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad2f2c-6bb0-4757-b0fa-02e7222cc6ec None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:29.051400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:29.126032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 426/842] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:29.127568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:29.144129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad2f2c-6bb0-4757-b0fa-02e7222cc6ec None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:29.185953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1fe8e-2737-4d26-9b42-28494d6b651b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:29.186559 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fe8e-2737-4d26-9b42-28494d6b651b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:29.186834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fe8e-2737-4d26-9b42-28494d6b651b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:29.187132 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fe8e-2737-4d26-9b42-28494d6b651b None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.187816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1fe8e-2737-4d26-9b42-28494d6b651b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:29.188116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1fe8e-2737-4d26-9b42-28494d6b651b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:29.189478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1fe8e-2737-4d26-9b42-28494d6b651b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:29.200620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 417/843] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:29.201598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b1cee-bca0-4f0a-9949-4a2eb098bf97 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:29.201879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b1cee-bca0-4f0a-9949-4a2eb098bf97 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:29.202149 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b1cee-bca0-4f0a-9949-4a2eb098bf97 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.203582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 427/844] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:10:29.208214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1fe8e-2737-4d26-9b42-28494d6b651b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:29.208713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ca3eb-ba1b-415b-8ed4-458e837c6abf None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:29.209200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ca3eb-ba1b-415b-8ed4-458e837c6abf None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:29.209660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ca3eb-ba1b-415b-8ed4-458e837c6abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.228779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-387ca3eb-ba1b-415b-8ed4-458e837c6abf None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:29.230350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-387ca3eb-ba1b-415b-8ed4-458e837c6abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:29.308064 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 428/845] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:29.326825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1fe8e-2737-4d26-9b42-28494d6b651b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:29.361441 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2071c2-d968-406c-9942-2c4692675a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:29.362063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2071c2-d968-406c-9942-2c4692675a8f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:29.362334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2071c2-d968-406c-9942-2c4692675a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:29.362598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2071c2-d968-406c-9942-2c4692675a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.373614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2071c2-d968-406c-9942-2c4692675a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:29.375094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2071c2-d968-406c-9942-2c4692675a8f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:29.381573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-0c2071c2-d968-406c-9942-2c4692675a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:29.389586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:29.390546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:29.391469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:29.391469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:29.391469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:29.391469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 418/846] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:29.398273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ca3eb-ba1b-415b-8ed4-458e837c6abf None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:29.403392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2071c2-d968-406c-9942-2c4692675a8f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:29.513768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2071c2-d968-406c-9942-2c4692675a8f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:29.521140 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:29.539828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ca3eb-ba1b-415b-8ed4-458e837c6abf None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:29.553845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe03308-f550-4d11-9674-b53386291fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:29.554468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe03308-f550-4d11-9674-b53386291fda None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:29.554742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe03308-f550-4d11-9674-b53386291fda None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:29.555008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe03308-f550-4d11-9674-b53386291fda None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.565754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe03308-f550-4d11-9674-b53386291fda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-0fe03308-f550-4d11-9674-b53386291fda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:29.570852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:29.572044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:29.573104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 419/847] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:10:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:29.580157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe03308-f550-4d11-9674-b53386291fda None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:29.588691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de79a-2014-4a0b-92e2-a63b72f62da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:29.589414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de79a-2014-4a0b-92e2-a63b72f62da5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:29.589782 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de79a-2014-4a0b-92e2-a63b72f62da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:29.590143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de79a-2014-4a0b-92e2-a63b72f62da5 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.591131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984de79a-2014-4a0b-92e2-a63b72f62da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:29.591869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984de79a-2014-4a0b-92e2-a63b72f62da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:29.594867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984de79a-2014-4a0b-92e2-a63b72f62da5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:29.606871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 429/848] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:29.612273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de79a-2014-4a0b-92e2-a63b72f62da5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:29.643712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:29.686506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe03308-f550-4d11-9674-b53386291fda None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:29.720047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de79a-2014-4a0b-92e2-a63b72f62da5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:29.725811 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d6283-cffe-4f01-a8bc-105b7b142f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:29.726407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d6283-cffe-4f01-a8bc-105b7b142f13 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:29.726682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d6283-cffe-4f01-a8bc-105b7b142f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:29.726948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d6283-cffe-4f01-a8bc-105b7b142f13 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.727634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d6283-cffe-4f01-a8bc-105b7b142f13 None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:29.727935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d6283-cffe-4f01-a8bc-105b7b142f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:29.729310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d6283-cffe-4f01-a8bc-105b7b142f13 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:29.742899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 420/849] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:10:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:29.743779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d6283-cffe-4f01-a8bc-105b7b142f13 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:29.755696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881ad28-1d01-4aea-a306-15d8447374d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:29.756324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881ad28-1d01-4aea-a306-15d8447374d7 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:29.756764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881ad28-1d01-4aea-a306-15d8447374d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:29.756873 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881ad28-1d01-4aea-a306-15d8447374d7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.764394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881ad28-1d01-4aea-a306-15d8447374d7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:29.766328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881ad28-1d01-4aea-a306-15d8447374d7 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-1881ad28-1d01-4aea-a306-15d8447374d7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:29.777463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:29.778410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 430/850] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:29.779374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881ad28-1d01-4aea-a306-15d8447374d7 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:29.794758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:29.858698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d6283-cffe-4f01-a8bc-105b7b142f13 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:29.879755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:29.889191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881ad28-1d01-4aea-a306-15d8447374d7 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:29.894209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a2e97-53e6-49a2-9694-d51c22eb2a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:29.894836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a2e97-53e6-49a2-9694-d51c22eb2a6b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:29.895114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a2e97-53e6-49a2-9694-d51c22eb2a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:29.895385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a2e97-53e6-49a2-9694-d51c22eb2a6b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.896002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a2e97-53e6-49a2-9694-d51c22eb2a6b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:29.897361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a2e97-53e6-49a2-9694-d51c22eb2a6b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-666a2e97-53e6-49a2-9694-d51c22eb2a6b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:29.906678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:29.907629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:10:29.908543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:29.913699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 421/851] 10.222.0.62 () {56 vars in 1132 bytes} [Mon Sep 5 12:10:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:29.914231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a2e97-53e6-49a2-9694-d51c22eb2a6b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:29.926965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a410926-b6fe-4657-bf4f-38c8fdc94e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:29.927643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410926-b6fe-4657-bf4f-38c8fdc94e66 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:29.927927 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410926-b6fe-4657-bf4f-38c8fdc94e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:29.928200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410926-b6fe-4657-bf4f-38c8fdc94e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:29.939633 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410926-b6fe-4657-bf4f-38c8fdc94e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:29.941058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410926-b6fe-4657-bf4f-38c8fdc94e66 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-1a410926-b6fe-4657-bf4f-38c8fdc94e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:29.957540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:29.958699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:29.959675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:29.959675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:29.959675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:29.959675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 431/852] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:29.959904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a410926-b6fe-4657-bf4f-38c8fdc94e66 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:30.026967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a2e97-53e6-49a2-9694-d51c22eb2a6b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:30.028319 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:30.071615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d9b2a-d500-4fb2-86ef-e728b1e25006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:30.072448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9b2a-d500-4fb2-86ef-e728b1e25006 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:30.072867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9b2a-d500-4fb2-86ef-e728b1e25006 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:30.073346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9b2a-d500-4fb2-86ef-e728b1e25006 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:30.074317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d9b2a-d500-4fb2-86ef-e728b1e25006 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:30.074768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d9b2a-d500-4fb2-86ef-e728b1e25006 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:30.076371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d9b2a-d500-4fb2-86ef-e728b1e25006 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:30.076583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a410926-b6fe-4657-bf4f-38c8fdc94e66 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:30.096020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 422/853] 10.222.0.62 () {56 vars in 1135 bytes} [Mon Sep 5 12:10:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:30.097213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d9b2a-d500-4fb2-86ef-e728b1e25006 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:30.118575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77440f-ed29-4f0a-b063-8af8125de2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:30.119111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77440f-ed29-4f0a-b063-8af8125de2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:30.119384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77440f-ed29-4f0a-b063-8af8125de2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:30.119683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77440f-ed29-4f0a-b063-8af8125de2e5 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:30.120315 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77440f-ed29-4f0a-b063-8af8125de2e5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:30.120616 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77440f-ed29-4f0a-b063-8af8125de2e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:30.121972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77440f-ed29-4f0a-b063-8af8125de2e5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:30.126570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 432/854] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:30.127345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77440f-ed29-4f0a-b063-8af8125de2e5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:30.147585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:30.208679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d9b2a-d500-4fb2-86ef-e728b1e25006 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:30.236424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77440f-ed29-4f0a-b063-8af8125de2e5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:30.244825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc1572-a4d8-42c5-a272-2944448b70b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:30.245422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1572-a4d8-42c5-a272-2944448b70b7 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:30.245701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1572-a4d8-42c5-a272-2944448b70b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:30.245958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1572-a4d8-42c5-a272-2944448b70b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:30.246593 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc1572-a4d8-42c5-a272-2944448b70b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:30.246888 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc1572-a4d8-42c5-a272-2944448b70b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:30.248269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc1572-a4d8-42c5-a272-2944448b70b7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:30.262725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:30.264977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 423/855] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:30.266245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc1572-a4d8-42c5-a272-2944448b70b7 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:30.272546 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e8550-18a7-452b-8473-5450c50ba4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:30.273215 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e8550-18a7-452b-8473-5450c50ba4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:30.273498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e8550-18a7-452b-8473-5450c50ba4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:30.273772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e8550-18a7-452b-8473-5450c50ba4f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:30.347483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e8550-18a7-452b-8473-5450c50ba4f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:30.349049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e8550-18a7-452b-8473-5450c50ba4f8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:30.369203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc1572-a4d8-42c5-a272-2944448b70b7 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:30.370635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:30.405973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffa5a2-48e5-4343-a0af-b2d484934982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:30.406780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffa5a2-48e5-4343-a0af-b2d484934982 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:30.407069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffa5a2-48e5-4343-a0af-b2d484934982 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:30.407346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffa5a2-48e5-4343-a0af-b2d484934982 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:30.408018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffa5a2-48e5-4343-a0af-b2d484934982 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:30.408335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffa5a2-48e5-4343-a0af-b2d484934982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:30.409694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffa5a2-48e5-4343-a0af-b2d484934982 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:30.419589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 424/856] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:30.419589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffa5a2-48e5-4343-a0af-b2d484934982 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:30.482179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 433/857] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:10:30.488698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e8550-18a7-452b-8473-5450c50ba4f8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:30.522140 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:30.558022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffa5a2-48e5-4343-a0af-b2d484934982 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:30.601607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e856a-dce2-40c6-9c62-e72b29b4db6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:30.602250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e856a-dce2-40c6-9c62-e72b29b4db6a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:30.602529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e856a-dce2-40c6-9c62-e72b29b4db6a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:30.602799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e856a-dce2-40c6-9c62-e72b29b4db6a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:30.608091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e8550-18a7-452b-8473-5450c50ba4f8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:30.615339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e856a-dce2-40c6-9c62-e72b29b4db6a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:30.617675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e856a-dce2-40c6-9c62-e72b29b4db6a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-e31e856a-dce2-40c6-9c62-e72b29b4db6a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:30.635046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:30.636051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:30.637003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:30.637003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:30.637003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:30.637003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:30.637003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:30.637003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:30.637003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:30.637003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:30.637003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:30.637003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:30.639719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 425/858] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Sep 5 12:10:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:30.647582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e856a-dce2-40c6-9c62-e72b29b4db6a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:30.652987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90edcf-3c95-405a-8770-8d83873fa2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:30.653884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90edcf-3c95-405a-8770-8d83873fa2fc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:30.654371 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90edcf-3c95-405a-8770-8d83873fa2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:30.654831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90edcf-3c95-405a-8770-8d83873fa2fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:30.670076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90edcf-3c95-405a-8770-8d83873fa2fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:30.672987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90edcf-3c95-405a-8770-8d83873fa2fc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:30.676047 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-4e90edcf-3c95-405a-8770-8d83873fa2fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:30.693235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:30.694226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:30.695137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:30.695137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:30.695137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:30.697544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 434/859] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:30.703373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90edcf-3c95-405a-8770-8d83873fa2fc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:30.761772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e856a-dce2-40c6-9c62-e72b29b4db6a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:30.802166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:30.809078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138985c-272c-456c-9d18-6b96024f6567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:30.810237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138985c-272c-456c-9d18-6b96024f6567 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:30.810749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138985c-272c-456c-9d18-6b96024f6567 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:30.811220 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138985c-272c-456c-9d18-6b96024f6567 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:30.812303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138985c-272c-456c-9d18-6b96024f6567 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:30.813202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138985c-272c-456c-9d18-6b96024f6567 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:30.815288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138985c-272c-456c-9d18-6b96024f6567 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:30.828774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90edcf-3c95-405a-8770-8d83873fa2fc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:30.838696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 426/860] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Sep 5 12:10:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:30.839706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138985c-272c-456c-9d18-6b96024f6567 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:30.873292 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e856891-7452-4161-8572-01135c957450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:30.874411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856891-7452-4161-8572-01135c957450 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:30.875282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856891-7452-4161-8572-01135c957450 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:30.876060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856891-7452-4161-8572-01135c957450 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:30.876911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e856891-7452-4161-8572-01135c957450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:30.877412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e856891-7452-4161-8572-01135c957450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:30.879977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e856891-7452-4161-8572-01135c957450 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:30.899752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 435/861] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:30.901332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e856891-7452-4161-8572-01135c957450 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:30.933500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:30.950156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138985c-272c-456c-9d18-6b96024f6567 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:30.995839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dd4c8-7373-4cf5-b305-1db3d271c2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:30.996447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dd4c8-7373-4cf5-b305-1db3d271c2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:30.996728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dd4c8-7373-4cf5-b305-1db3d271c2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:30.996980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dd4c8-7373-4cf5-b305-1db3d271c2c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:31.010422 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e856891-7452-4161-8572-01135c957450 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:31.048454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521b548-e3eb-4dfd-8184-e10dc6c1414a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:31.049417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521b548-e3eb-4dfd-8184-e10dc6c1414a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:31.049758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521b548-e3eb-4dfd-8184-e10dc6c1414a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:31.050094 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521b548-e3eb-4dfd-8184-e10dc6c1414a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:31.057172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521b548-e3eb-4dfd-8184-e10dc6c1414a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:31.058664 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521b548-e3eb-4dfd-8184-e10dc6c1414a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:31.060617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-4521b548-e3eb-4dfd-8184-e10dc6c1414a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:31.070794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:31.071796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:31.072723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:31.072723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:31.072723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:31.072723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:31.072723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:31.072723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:31.072723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:31.072723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:31.076915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 436/862] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:31.078646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369acc6-5e14-41d6-bc76-5e103f745d88 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:31.078935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369acc6-5e14-41d6-bc76-5e103f745d88 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:31.079204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369acc6-5e14-41d6-bc76-5e103f745d88 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:31.080847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 437/863] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:31.081509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369acc6-5e14-41d6-bc76-5e103f745d88 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:31.085393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dd4c8-7373-4cf5-b305-1db3d271c2c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:31.086976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dd4c8-7373-4cf5-b305-1db3d271c2c4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:31.176308 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:31.196661 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 427/864] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:10:31.196908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369acc6-5e14-41d6-bc76-5e103f745d88 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:31.197315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a43c6-59fb-4d2c-ba78-80aca5e5e656 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:31.197315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a43c6-59fb-4d2c-ba78-80aca5e5e656 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:31.197315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a43c6-59fb-4d2c-ba78-80aca5e5e656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:31.213863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7d8a43c6-59fb-4d2c-ba78-80aca5e5e656 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:31.215053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7d8a43c6-59fb-4d2c-ba78-80aca5e5e656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:31.233091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa30c9-b663-4095-9e43-61d01668dd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:31.233700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa30c9-b663-4095-9e43-61d01668dd23 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:31.233977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa30c9-b663-4095-9e43-61d01668dd23 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:31.234275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa30c9-b663-4095-9e43-61d01668dd23 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:31.234965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa30c9-b663-4095-9e43-61d01668dd23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:31.235266 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa30c9-b663-4095-9e43-61d01668dd23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:31.236707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa30c9-b663-4095-9e43-61d01668dd23 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:31.248779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 438/865] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:31.249560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa30c9-b663-4095-9e43-61d01668dd23 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:31.289128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 428/866] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:31.290184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a43c6-59fb-4d2c-ba78-80aca5e5e656 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:31.317653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:31.362892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa30c9-b663-4095-9e43-61d01668dd23 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:31.400156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392c6c4-99e4-4440-9745-480cb8e1ca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:31.400853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392c6c4-99e4-4440-9745-480cb8e1ca89 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:31.401059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392c6c4-99e4-4440-9745-480cb8e1ca89 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:31.401372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392c6c4-99e4-4440-9745-480cb8e1ca89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:31.412356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392c6c4-99e4-4440-9745-480cb8e1ca89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:31.414009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392c6c4-99e4-4440-9745-480cb8e1ca89 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:31.420595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a43c6-59fb-4d2c-ba78-80aca5e5e656 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-6392c6c4-99e4-4440-9745-480cb8e1ca89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:31.430362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:31.431295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:31.432251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:31.432251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:31.432251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:31.436183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 439/867] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:31.438177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392c6c4-99e4-4440-9745-480cb8e1ca89 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:31.452016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:31.468564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128be2e-cce7-4001-85b1-a938f86113bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:31.470129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128be2e-cce7-4001-85b1-a938f86113bb None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:31.470614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128be2e-cce7-4001-85b1-a938f86113bb None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:31.471295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128be2e-cce7-4001-85b1-a938f86113bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:31.472507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128be2e-cce7-4001-85b1-a938f86113bb None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=b48aa41ff66b42329926bc42379d9800) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:31.473386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128be2e-cce7-4001-85b1-a938f86113bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:31.475274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128be2e-cce7-4001-85b1-a938f86113bb None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:31.483927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 429/868] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:31 2022] GET /identity/v3/role_assignments?user.id=b48aa41ff66b42329926bc42379d9800&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=705907b9bdec47cebf88b868a6a56356 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:31.559054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392c6c4-99e4-4440-9745-480cb8e1ca89 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:31.572972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:31.579421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128be2e-cce7-4001-85b1-a938f86113bb None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:31.600598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072da40d-e1dd-429f-8469-3e73a2a68931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:31.601371 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072da40d-e1dd-429f-8469-3e73a2a68931 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:31.601644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072da40d-e1dd-429f-8469-3e73a2a68931 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:31.601924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072da40d-e1dd-429f-8469-3e73a2a68931 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:31.602593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072da40d-e1dd-429f-8469-3e73a2a68931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:31.602899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072da40d-e1dd-429f-8469-3e73a2a68931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:31.604460 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072da40d-e1dd-429f-8469-3e73a2a68931 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:31.619016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 440/869] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:31.624557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072da40d-e1dd-429f-8469-3e73a2a68931 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:31.703792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128be2e-cce7-4001-85b1-a938f86113bb None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:31.743671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e03f0b-8489-48a0-9f2a-1ea906b7170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:31.744884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e03f0b-8489-48a0-9f2a-1ea906b7170e None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:31.745780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e03f0b-8489-48a0-9f2a-1ea906b7170e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:31.746551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e03f0b-8489-48a0-9f2a-1ea906b7170e None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:31.747678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e03f0b-8489-48a0-9f2a-1ea906b7170e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:31.750690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e03f0b-8489-48a0-9f2a-1ea906b7170e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:31.754929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072da40d-e1dd-429f-8469-3e73a2a68931 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:31.790974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c2869-fe5a-4550-a8cf-a925b80d9dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:31.791958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c2869-fe5a-4550-a8cf-a925b80d9dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:31.792239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c2869-fe5a-4550-a8cf-a925b80d9dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:31.792506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c2869-fe5a-4550-a8cf-a925b80d9dc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:31.800366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c2869-fe5a-4550-a8cf-a925b80d9dc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:31.801964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c2869-fe5a-4550-a8cf-a925b80d9dc6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-d49c2869-fe5a-4550-a8cf-a925b80d9dc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:31.814455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:31.815326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:31.816263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:31.816263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:31.816263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:31.816263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:31.816263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:31.816263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:31.816263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:31.816263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:31.817212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 441/870] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:31.821748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c2869-fe5a-4550-a8cf-a925b80d9dc6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:31.845873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 430/871] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:10:31 2022] POST /identity/v3/services => generated 251 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:31.864274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:31.950261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c2869-fe5a-4550-a8cf-a925b80d9dc6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:31.998363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0106a96-f77d-479a-8eda-ed7bf7937016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:31.998903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0106a96-f77d-479a-8eda-ed7bf7937016 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:31.999257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0106a96-f77d-479a-8eda-ed7bf7937016 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:31.999501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0106a96-f77d-479a-8eda-ed7bf7937016 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:32.000836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0106a96-f77d-479a-8eda-ed7bf7937016 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:32.001096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0106a96-f77d-479a-8eda-ed7bf7937016 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:32.002606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0106a96-f77d-479a-8eda-ed7bf7937016 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:32.019645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 442/872] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:32.022730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e03f0b-8489-48a0-9f2a-1ea906b7170e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:32.076176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:32.140849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e03f0b-8489-48a0-9f2a-1ea906b7170e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:32.177019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad441c-917c-4350-a420-55d2c3a26a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:32.178359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad441c-917c-4350-a420-55d2c3a26a04 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:32.178840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad441c-917c-4350-a420-55d2c3a26a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:32.179301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad441c-917c-4350-a420-55d2c3a26a04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:32.180302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad441c-917c-4350-a420-55d2c3a26a04 None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=81b72e82b4a34ab6af29ef440d19f6aa) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:32.180782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad441c-917c-4350-a420-55d2c3a26a04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:32.182402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad441c-917c-4350-a420-55d2c3a26a04 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:32.189749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 431/873] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:32 2022] GET /identity/v3/role_assignments?user.id=81b72e82b4a34ab6af29ef440d19f6aa&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:32.427734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959531e-4dfe-47b8-a0a4-016f226f7515 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:32.428267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959531e-4dfe-47b8-a0a4-016f226f7515 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:32.428267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959531e-4dfe-47b8-a0a4-016f226f7515 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:32.430014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 443/874] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:32.435281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684d41c-c022-4b02-bc08-fd07205edcfb None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:32.435598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684d41c-c022-4b02-bc08-fd07205edcfb None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:32.435881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684d41c-c022-4b02-bc08-fd07205edcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:32.455117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1684d41c-c022-4b02-bc08-fd07205edcfb None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:32.456358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1684d41c-c022-4b02-bc08-fd07205edcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:32.526749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 432/875] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:32.799602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959531e-4dfe-47b8-a0a4-016f226f7515 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:32.893206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:32.910653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959531e-4dfe-47b8-a0a4-016f226f7515 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:32.946827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1211c-2e2a-4185-8e59-22234631c5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:32.947438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1211c-2e2a-4185-8e59-22234631c5ba None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:32.947767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1211c-2e2a-4185-8e59-22234631c5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:32.948028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1211c-2e2a-4185-8e59-22234631c5ba None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:32.948694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1211c-2e2a-4185-8e59-22234631c5ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:32.950055 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1211c-2e2a-4185-8e59-22234631c5ba None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:33.038458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 444/876] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:10:32 2022] POST /identity/v3/services => generated 269 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:33.486165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a5006-e1ab-4762-a560-498e9edd8e7d None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:33.486685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a5006-e1ab-4762-a560-498e9edd8e7d None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:33.486685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a5006-e1ab-4762-a560-498e9edd8e7d None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:33.488281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 433/877] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:33.493651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd79ebf-628d-4410-9999-4e111dd6dc4b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:33.494048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd79ebf-628d-4410-9999-4e111dd6dc4b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:33.494235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd79ebf-628d-4410-9999-4e111dd6dc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:33.513339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fdd79ebf-628d-4410-9999-4e111dd6dc4b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:33.514514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fdd79ebf-628d-4410-9999-4e111dd6dc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:33.591612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 445/878] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:33.861746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a5006-e1ab-4762-a560-498e9edd8e7d None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:33.893784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:33.915969 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb37046-5177-43f4-9542-d5e4816644e0 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:33.916251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb37046-5177-43f4-9542-d5e4816644e0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:33.916523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb37046-5177-43f4-9542-d5e4816644e0 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:33.918298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 446/879] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:33.923793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5c7b4-547b-444c-9a51-d474fb7233e4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:33.924074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5c7b4-547b-444c-9a51-d474fb7233e4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:33.924349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5c7b4-547b-444c-9a51-d474fb7233e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:33.946566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-83c5c7b4-547b-444c-9a51-d474fb7233e4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:33.948277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-83c5c7b4-547b-444c-9a51-d474fb7233e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:33.977312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a5006-e1ab-4762-a560-498e9edd8e7d None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:34.014600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4193e1-74b3-4447-ab2c-ff9519fb34cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:34.015225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4193e1-74b3-4447-ab2c-ff9519fb34cb None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:34.016064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4193e1-74b3-4447-ab2c-ff9519fb34cb None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:34.016340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4193e1-74b3-4447-ab2c-ff9519fb34cb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:34.016918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4193e1-74b3-4447-ab2c-ff9519fb34cb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:34.018281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4193e1-74b3-4447-ab2c-ff9519fb34cb None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-fa4193e1-74b3-4447-ab2c-ff9519fb34cb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:34.027253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:34.028294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:34.029491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 434/880] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:34.034740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4193e1-74b3-4447-ab2c-ff9519fb34cb None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:34.051696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 447/881] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:34.155797 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:34.163942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4193e1-74b3-4447-ab2c-ff9519fb34cb None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:34.193220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063753a-bd43-49c4-b14c-bca8257fbdb3 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:34.193517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063753a-bd43-49c4-b14c-bca8257fbdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:34.193800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063753a-bd43-49c4-b14c-bca8257fbdb3 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:34.198870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 448/882] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:10:34.200290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c2258-2b33-4e64-a1a9-c23da7c2350a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:34.200574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c2258-2b33-4e64-a1a9-c23da7c2350a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:34.200851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c2258-2b33-4e64-a1a9-c23da7c2350a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:34.201435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff7614-4a4a-4970-82b4-69aa179d7aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:34.202197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff7614-4a4a-4970-82b4-69aa179d7aea None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:34.202612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff7614-4a4a-4970-82b4-69aa179d7aea None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:34.203605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff7614-4a4a-4970-82b4-69aa179d7aea None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:34.204371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff7614-4a4a-4970-82b4-69aa179d7aea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:34.204911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff7614-4a4a-4970-82b4-69aa179d7aea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:34.206605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff7614-4a4a-4970-82b4-69aa179d7aea None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:34.208360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:34.213854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 435/883] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Sep 5 12:10:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:34.220641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff7614-4a4a-4970-82b4-69aa179d7aea None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:34.225532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-216c2258-2b33-4e64-a1a9-c23da7c2350a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:34.226727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-216c2258-2b33-4e64-a1a9-c23da7c2350a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:34.303659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 449/884] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:34.324635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff7614-4a4a-4970-82b4-69aa179d7aea None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:34.351997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c2258-2b33-4e64-a1a9-c23da7c2350a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:34.360499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afcaec-349c-4045-839a-38cc4d52d328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:34.361392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afcaec-349c-4045-839a-38cc4d52d328 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:34.361812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afcaec-349c-4045-839a-38cc4d52d328 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:34.362219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afcaec-349c-4045-839a-38cc4d52d328 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:34.362997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afcaec-349c-4045-839a-38cc4d52d328 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:34.363562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afcaec-349c-4045-839a-38cc4d52d328 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:34.365163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afcaec-349c-4045-839a-38cc4d52d328 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:34.370511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 436/885] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Sep 5 12:10:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:34.472370 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c2258-2b33-4e64-a1a9-c23da7c2350a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:34.505298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:34.510871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ef870-cc53-45a6-b38d-e18f6e42b715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:34.511721 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ef870-cc53-45a6-b38d-e18f6e42b715 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:34.512005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ef870-cc53-45a6-b38d-e18f6e42b715 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:34.512279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ef870-cc53-45a6-b38d-e18f6e42b715 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:34.517128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ef870-cc53-45a6-b38d-e18f6e42b715 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:34.517128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ef870-cc53-45a6-b38d-e18f6e42b715 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-383ef870-cc53-45a6-b38d-e18f6e42b715 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:34.524252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:34.525219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:34.526052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 450/886] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:10:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:34.531835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afcaec-349c-4045-839a-38cc4d52d328 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:34.564674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:34.598994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ef870-cc53-45a6-b38d-e18f6e42b715 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:34.646128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afcaec-349c-4045-839a-38cc4d52d328 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:34.681682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424b2e1-3bad-4ea7-acf5-393b753a3468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:34.682582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424b2e1-3bad-4ea7-acf5-393b753a3468 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:34.683012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424b2e1-3bad-4ea7-acf5-393b753a3468 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:34.683431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424b2e1-3bad-4ea7-acf5-393b753a3468 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:34.684282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424b2e1-3bad-4ea7-acf5-393b753a3468 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:34.684785 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424b2e1-3bad-4ea7-acf5-393b753a3468 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:34.686349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424b2e1-3bad-4ea7-acf5-393b753a3468 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:34.692403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 437/887] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:10:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:34.698082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424b2e1-3bad-4ea7-acf5-393b753a3468 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:34.712218 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ef870-cc53-45a6-b38d-e18f6e42b715 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:34.750553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b406f9-9601-4f64-9cfa-e01614526c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:34.751089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b406f9-9601-4f64-9cfa-e01614526c92 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:34.751441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b406f9-9601-4f64-9cfa-e01614526c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:34.751962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b406f9-9601-4f64-9cfa-e01614526c92 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:34.763239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b406f9-9601-4f64-9cfa-e01614526c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e8b406f9-9601-4f64-9cfa-e01614526c92 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:34.768953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:34.769898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:10:34.770756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:34.774960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 451/888] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Sep 5 12:10:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:34.776298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b406f9-9601-4f64-9cfa-e01614526c92 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:34.797359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:34.809711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424b2e1-3bad-4ea7-acf5-393b753a3468 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:34.839448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:34.847280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb45f6-766d-46db-b221-e3c1f812d044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:34.848180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb45f6-766d-46db-b221-e3c1f812d044 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:34.848636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb45f6-766d-46db-b221-e3c1f812d044 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:34.849095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb45f6-766d-46db-b221-e3c1f812d044 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:34.849929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb45f6-766d-46db-b221-e3c1f812d044 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:34.850400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb45f6-766d-46db-b221-e3c1f812d044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:34.851979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb45f6-766d-46db-b221-e3c1f812d044 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:34.863222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 438/889] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:10:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:34.864512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb45f6-766d-46db-b221-e3c1f812d044 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:34.899062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b406f9-9601-4f64-9cfa-e01614526c92 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:34.952491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49205588-c745-492f-91d8-cd5a6dd4fc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:34.953145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49205588-c745-492f-91d8-cd5a6dd4fc67 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:34.953420 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49205588-c745-492f-91d8-cd5a6dd4fc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:34.953691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49205588-c745-492f-91d8-cd5a6dd4fc67 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:34.954369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49205588-c745-492f-91d8-cd5a6dd4fc67 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:34.954679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49205588-c745-492f-91d8-cd5a6dd4fc67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:34.956610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49205588-c745-492f-91d8-cd5a6dd4fc67 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:34.971694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 452/890] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Sep 5 12:10:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:34.974470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49205588-c745-492f-91d8-cd5a6dd4fc67 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:34.986286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb45f6-766d-46db-b221-e3c1f812d044 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:35.023197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01280d-4cf9-4ac0-b1a6-c80f81433e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:35.023920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01280d-4cf9-4ac0-b1a6-c80f81433e10 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.024201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01280d-4cf9-4ac0-b1a6-c80f81433e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.024481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01280d-4cf9-4ac0-b1a6-c80f81433e10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.025262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01280d-4cf9-4ac0-b1a6-c80f81433e10 None admin] RBAC: Adding query filter params (interface=public, service_id=89605f555718411fb9c1a9802642bbb7, region_id=RegionOne) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:35.025579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01280d-4cf9-4ac0-b1a6-c80f81433e10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:35.026943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01280d-4cf9-4ac0-b1a6-c80f81433e10 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:35.032192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 439/891] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Sep 5 12:10:34 2022] GET /identity/v3/endpoints?service_id=89605f555718411fb9c1a9802642bbb7&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:35.033265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8772d4-86ea-4a0d-9059-8ed41b45754e None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.033535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8772d4-86ea-4a0d-9059-8ed41b45754e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.033785 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8772d4-86ea-4a0d-9059-8ed41b45754e None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.035639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 440/892] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:35.038013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8772d4-86ea-4a0d-9059-8ed41b45754e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:35.057354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:35.095204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49205588-c745-492f-91d8-cd5a6dd4fc67 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:35.113965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:35.133284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097e1dd-2720-447c-960f-1334eb96c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:35.133904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097e1dd-2720-447c-960f-1334eb96c356 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.134180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097e1dd-2720-447c-960f-1334eb96c356 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.134561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097e1dd-2720-447c-960f-1334eb96c356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.148816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097e1dd-2720-447c-960f-1334eb96c356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:35.150239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097e1dd-2720-447c-960f-1334eb96c356 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:35.152857 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8772d4-86ea-4a0d-9059-8ed41b45754e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-3097e1dd-2720-447c-960f-1334eb96c356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:35.165246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:35.166195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:35.167095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:35.167095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:35.167095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:35.167095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 453/893] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:35.167920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06302993-aaf4-4dae-8334-91f003cb1446 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.168092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06302993-aaf4-4dae-8334-91f003cb1446 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.168333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06302993-aaf4-4dae-8334-91f003cb1446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.187640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-06302993-aaf4-4dae-8334-91f003cb1446 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:35.188814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-06302993-aaf4-4dae-8334-91f003cb1446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:35.194218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8ed38-6b5a-423e-8782-24463752fff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:35.194826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8ed38-6b5a-423e-8782-24463752fff0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.195102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8ed38-6b5a-423e-8782-24463752fff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.195371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8ed38-6b5a-423e-8782-24463752fff0 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.196001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8ed38-6b5a-423e-8782-24463752fff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:35.197380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8ed38-6b5a-423e-8782-24463752fff0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:35.211577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 441/894] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Sep 5 12:10:35 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:10:35.211577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8ed38-6b5a-423e-8782-24463752fff0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:35.277289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 454/895] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:35.314822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:35.335258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8ed38-6b5a-423e-8782-24463752fff0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:35.374249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427ecc7-eba5-49c1-809e-08aabb90f8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:35.374888 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ecc7-eba5-49c1-809e-08aabb90f8ba None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.375164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ecc7-eba5-49c1-809e-08aabb90f8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.375432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ecc7-eba5-49c1-809e-08aabb90f8ba None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.376106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427ecc7-eba5-49c1-809e-08aabb90f8ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:35.376403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427ecc7-eba5-49c1-809e-08aabb90f8ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:35.377771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427ecc7-eba5-49c1-809e-08aabb90f8ba None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:35.389867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 442/896] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:35.395377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06302993-aaf4-4dae-8334-91f003cb1446 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:35.499946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:35.503552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06302993-aaf4-4dae-8334-91f003cb1446 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:35.540771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5bd1b1-1638-4f37-99f3-3138e0a9a890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:35.542191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bd1b1-1638-4f37-99f3-3138e0a9a890 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.542690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bd1b1-1638-4f37-99f3-3138e0a9a890 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.543181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bd1b1-1638-4f37-99f3-3138e0a9a890 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.550882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427ecc7-eba5-49c1-809e-08aabb90f8ba None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:35.551791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5bd1b1-1638-4f37-99f3-3138e0a9a890 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:35.553639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5bd1b1-1638-4f37-99f3-3138e0a9a890 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-db5bd1b1-1638-4f37-99f3-3138e0a9a890 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:35.566409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:35.567384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:35.568384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:35.568384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:35.568384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:35.568384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:35.568384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:35.568384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:35.568384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:35.568384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:35.568384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:35.568384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:35.570932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 455/897] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Sep 5 12:10:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:35.577344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5bd1b1-1638-4f37-99f3-3138e0a9a890 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:35.674521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427ecc7-eba5-49c1-809e-08aabb90f8ba None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:35.695914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5bd1b1-1638-4f37-99f3-3138e0a9a890 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:35.717274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fb539-3a54-4736-a0ea-683da142ae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:35.718184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fb539-3a54-4736-a0ea-683da142ae6f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.718463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fb539-3a54-4736-a0ea-683da142ae6f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.719289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fb539-3a54-4736-a0ea-683da142ae6f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.719910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fb539-3a54-4736-a0ea-683da142ae6f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:35.721290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fb539-3a54-4736-a0ea-683da142ae6f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-295fb539-3a54-4736-a0ea-683da142ae6f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:35.733556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:35.734673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:10:35.735595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:35.739689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 443/898] 10.222.0.62 () {56 vars in 1132 bytes} [Mon Sep 5 12:10:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:35.740584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fb539-3a54-4736-a0ea-683da142ae6f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:35.741185 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1853bf-709e-4c30-ac4d-812d2c49af9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:35.742017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1853bf-709e-4c30-ac4d-812d2c49af9b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.742471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1853bf-709e-4c30-ac4d-812d2c49af9b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.742914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1853bf-709e-4c30-ac4d-812d2c49af9b None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.745040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:35.746608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1853bf-709e-4c30-ac4d-812d2c49af9b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:35.746999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1853bf-709e-4c30-ac4d-812d2c49af9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:35.748475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1853bf-709e-4c30-ac4d-812d2c49af9b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:35.772108 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 456/899] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Sep 5 12:10:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:35.773009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1853bf-709e-4c30-ac4d-812d2c49af9b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:35.795973 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:35.876082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fb539-3a54-4736-a0ea-683da142ae6f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:35.880621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1853bf-709e-4c30-ac4d-812d2c49af9b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:35.917638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909ede0-c05d-45c8-b546-eee6c0c9bd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:35.918398 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909ede0-c05d-45c8-b546-eee6c0c9bd88 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.918674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909ede0-c05d-45c8-b546-eee6c0c9bd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.918944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909ede0-c05d-45c8-b546-eee6c0c9bd88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.919977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc3c4a-75ae-45ba-8039-0a7ed984c7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:35.920829 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc3c4a-75ae-45ba-8039-0a7ed984c7ef None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:35.921319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc3c4a-75ae-45ba-8039-0a7ed984c7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:35.922607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc3c4a-75ae-45ba-8039-0a7ed984c7ef None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:35.924330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc3c4a-75ae-45ba-8039-0a7ed984c7ef None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:35.924860 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc3c4a-75ae-45ba-8039-0a7ed984c7ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:35.926446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc3c4a-75ae-45ba-8039-0a7ed984c7ef None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:35.937333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 444/900] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:35.938457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc3c4a-75ae-45ba-8039-0a7ed984c7ef None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:35.997975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909ede0-c05d-45c8-b546-eee6c0c9bd88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:35.999708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909ede0-c05d-45c8-b546-eee6c0c9bd88 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:36.023372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:36.032056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:36.066595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc3c4a-75ae-45ba-8039-0a7ed984c7ef None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:36.107424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5019c-e6e5-432d-905b-93a9108f0c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:36.108631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5019c-e6e5-432d-905b-93a9108f0c5c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:36.109113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5019c-e6e5-432d-905b-93a9108f0c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:36.110111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5019c-e6e5-432d-905b-93a9108f0c5c None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:36.111716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5019c-e6e5-432d-905b-93a9108f0c5c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:36.112228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5019c-e6e5-432d-905b-93a9108f0c5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:36.113822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5019c-e6e5-432d-905b-93a9108f0c5c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:36.123998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 445/901] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:36.125762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5019c-e6e5-432d-905b-93a9108f0c5c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:36.135306 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 457/902] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:35 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:10:36.136604 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909ede0-c05d-45c8-b546-eee6c0c9bd88 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:36.243948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5019c-e6e5-432d-905b-93a9108f0c5c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:36.267741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909ede0-c05d-45c8-b546-eee6c0c9bd88 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:36.282167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:36.289188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:36.290065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d1edb-5139-4302-a414-4b466abc0ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:36.290827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d1edb-5139-4302-a414-4b466abc0ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:36.291259 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d1edb-5139-4302-a414-4b466abc0ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:36.291693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d1edb-5139-4302-a414-4b466abc0ff9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:36.293061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d1edb-5139-4302-a414-4b466abc0ff9 None admin] RBAC: Adding query filter params (interface=public, service_id=ce6dcc23e31c4c21b3d463329ec98e57, region_id=RegionOne) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:36.293510 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d1edb-5139-4302-a414-4b466abc0ff9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:36.295347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d1edb-5139-4302-a414-4b466abc0ff9 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:36.303202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 446/903] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Sep 5 12:10:36 2022] GET /identity/v3/endpoints?service_id=ce6dcc23e31c4c21b3d463329ec98e57&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:36.311589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d1edb-5139-4302-a414-4b466abc0ff9 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:36.315052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34987b9-219a-4e1d-8af9-3e9ce5b1e6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:36.315793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34987b9-219a-4e1d-8af9-3e9ce5b1e6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:36.316071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34987b9-219a-4e1d-8af9-3e9ce5b1e6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:36.316343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34987b9-219a-4e1d-8af9-3e9ce5b1e6b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:36.327151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34987b9-219a-4e1d-8af9-3e9ce5b1e6b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:36.329345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34987b9-219a-4e1d-8af9-3e9ce5b1e6b0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-d34987b9-219a-4e1d-8af9-3e9ce5b1e6b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:36.345985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:36.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:36.347916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:36.347916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:36.347916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:36.347916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 458/904] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:36.348533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677da416-9a43-48fc-b474-ccf2522741e5 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:36.348818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677da416-9a43-48fc-b474-ccf2522741e5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:36.349092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677da416-9a43-48fc-b474-ccf2522741e5 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:36.354282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 459/905] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:36.354282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677da416-9a43-48fc-b474-ccf2522741e5 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:36.424560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d1edb-5139-4302-a414-4b466abc0ff9 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:36.463349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3eaa0-f173-4afa-a820-daaf92829b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:36.464118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3eaa0-f173-4afa-a820-daaf92829b34 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:36.464529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3eaa0-f173-4afa-a820-daaf92829b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:36.464899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3eaa0-f173-4afa-a820-daaf92829b34 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:36.465594 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3eaa0-f173-4afa-a820-daaf92829b34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:36.466150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677da416-9a43-48fc-b474-ccf2522741e5 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:36.467409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3eaa0-f173-4afa-a820-daaf92829b34 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:36.475518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 447/906] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Sep 5 12:10:36 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:10:36.476892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c9104-b04c-4393-b862-f25406d4da2b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:36.477316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c9104-b04c-4393-b862-f25406d4da2b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:36.477731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c9104-b04c-4393-b862-f25406d4da2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:36.497979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-5a9c9104-b04c-4393-b862-f25406d4da2b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:36.499570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5a9c9104-b04c-4393-b862-f25406d4da2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:36.519793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728dddc-92a3-4615-bb14-fd04e71477a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:36.520580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728dddc-92a3-4615-bb14-fd04e71477a5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:36.520959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728dddc-92a3-4615-bb14-fd04e71477a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:36.521732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728dddc-92a3-4615-bb14-fd04e71477a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:36.522528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728dddc-92a3-4615-bb14-fd04e71477a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:36.522940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728dddc-92a3-4615-bb14-fd04e71477a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:36.524483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728dddc-92a3-4615-bb14-fd04e71477a5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:36.542284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 460/907] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:36.548737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728dddc-92a3-4615-bb14-fd04e71477a5 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:36.576879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:36.585442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:36.595582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 448/908] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:36.670905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728dddc-92a3-4615-bb14-fd04e71477a5 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:36.707613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb2263-05d0-44c9-a0aa-589ccf6aa808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:36.708250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb2263-05d0-44c9-a0aa-589ccf6aa808 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:36.708533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb2263-05d0-44c9-a0aa-589ccf6aa808 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:36.708802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb2263-05d0-44c9-a0aa-589ccf6aa808 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:36.716093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb2263-05d0-44c9-a0aa-589ccf6aa808 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:36.717517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb2263-05d0-44c9-a0aa-589ccf6aa808 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-c8bb2263-05d0-44c9-a0aa-589ccf6aa808 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:36.729170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:36.730054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:36.730938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:36.730938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:36.730938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:36.730938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:36.730938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:36.730938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:36.730938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:36.730938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:36.730938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 461/909] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:36.737712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c9104-b04c-4393-b862-f25406d4da2b None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:36.844833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c9104-b04c-4393-b862-f25406d4da2b None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:36.862808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb2263-05d0-44c9-a0aa-589ccf6aa808 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:36.881033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff5165-047f-455f-b4cf-79270bc444a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:36.881745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff5165-047f-455f-b4cf-79270bc444a0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:36.882041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff5165-047f-455f-b4cf-79270bc444a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:36.882317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff5165-047f-455f-b4cf-79270bc444a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:36.883021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff5165-047f-455f-b4cf-79270bc444a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:36.883324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff5165-047f-455f-b4cf-79270bc444a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:36.884930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff5165-047f-455f-b4cf-79270bc444a0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:36.899590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 449/910] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:36.903984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff5165-047f-455f-b4cf-79270bc444a0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:36.970187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:36.973902 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:36.984213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb2263-05d0-44c9-a0aa-589ccf6aa808 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:37.025471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55305ebe-78b6-48e4-a7c5-baf3d3e45b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:37.026761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55305ebe-78b6-48e4-a7c5-baf3d3e45b92 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:37.027464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55305ebe-78b6-48e4-a7c5-baf3d3e45b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:37.027950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55305ebe-78b6-48e4-a7c5-baf3d3e45b92 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:37.028962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55305ebe-78b6-48e4-a7c5-baf3d3e45b92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:37.031119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55305ebe-78b6-48e4-a7c5-baf3d3e45b92 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:37.032893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff5165-047f-455f-b4cf-79270bc444a0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:37.068598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377ed13-d4c2-4a1a-9cc5-c7612b17a9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:37.069380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377ed13-d4c2-4a1a-9cc5-c7612b17a9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:37.069743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377ed13-d4c2-4a1a-9cc5-c7612b17a9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:37.070107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377ed13-d4c2-4a1a-9cc5-c7612b17a9c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:37.070922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377ed13-d4c2-4a1a-9cc5-c7612b17a9c5 None admin] RBAC: Adding query filter params (role.id=8b2491af21004040878bb0b8acb0194d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=81b72e82b4a34ab6af29ef440d19f6aa) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:37.071351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377ed13-d4c2-4a1a-9cc5-c7612b17a9c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:37.072897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377ed13-d4c2-4a1a-9cc5-c7612b17a9c5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:37.083927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 450/911] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:36 2022] GET /identity/v3/role_assignments?user.id=81b72e82b4a34ab6af29ef440d19f6aa&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=8b2491af21004040878bb0b8acb0194d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:37.113196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 462/912] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:10:36 2022] POST /identity/v3/services => generated 241 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:10:37.289671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962e37b-c83e-4fa7-a5cb-11d956195ed6 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:37.289955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962e37b-c83e-4fa7-a5cb-11d956195ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:37.290231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962e37b-c83e-4fa7-a5cb-11d956195ed6 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:37.291868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 451/913] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:37.296922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc39ca-671a-4d08-93c4-e8314d31bad7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:37.297212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc39ca-671a-4d08-93c4-e8314d31bad7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:37.297490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc39ca-671a-4d08-93c4-e8314d31bad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:37.318042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3cdc39ca-671a-4d08-93c4-e8314d31bad7 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:37.319239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3cdc39ca-671a-4d08-93c4-e8314d31bad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:37.398081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 463/914] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:37.679107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962e37b-c83e-4fa7-a5cb-11d956195ed6 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:37.785323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962e37b-c83e-4fa7-a5cb-11d956195ed6 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:37.799943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:37.820909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b602cd-fdc6-426a-a80a-a464d4b6da77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:37.821543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b602cd-fdc6-426a-a80a-a464d4b6da77 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:37.821823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b602cd-fdc6-426a-a80a-a464d4b6da77 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:37.822146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b602cd-fdc6-426a-a80a-a464d4b6da77 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:37.822730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b602cd-fdc6-426a-a80a-a464d4b6da77 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:37.824198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b602cd-fdc6-426a-a80a-a464d4b6da77 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-85b602cd-fdc6-426a-a80a-a464d4b6da77 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:37.833191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:37.834136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:37.834981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 452/915] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:10:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:37.840595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc39ca-671a-4d08-93c4-e8314d31bad7 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:37.920174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:37.945669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc39ca-671a-4d08-93c4-e8314d31bad7 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:37.980920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0de4e-b698-4c83-9daa-199102301a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:37.981525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0de4e-b698-4c83-9daa-199102301a32 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:37.981802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0de4e-b698-4c83-9daa-199102301a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:37.982072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0de4e-b698-4c83-9daa-199102301a32 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:37.982701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0de4e-b698-4c83-9daa-199102301a32 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:37.982998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0de4e-b698-4c83-9daa-199102301a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:37.984924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0de4e-b698-4c83-9daa-199102301a32 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:37.990081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 464/916] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:10:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:37.995346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b602cd-fdc6-426a-a80a-a464d4b6da77 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:38.100598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b602cd-fdc6-426a-a80a-a464d4b6da77 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:38.136031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f23ffa-ee23-45f1-b0c2-a85bc3db4d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:38.136649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f23ffa-ee23-45f1-b0c2-a85bc3db4d92 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:38.136925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f23ffa-ee23-45f1-b0c2-a85bc3db4d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:38.137198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f23ffa-ee23-45f1-b0c2-a85bc3db4d92 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:38.137822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f23ffa-ee23-45f1-b0c2-a85bc3db4d92 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:38.138121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f23ffa-ee23-45f1-b0c2-a85bc3db4d92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:38.139493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f23ffa-ee23-45f1-b0c2-a85bc3db4d92 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:38.144801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 453/917] 10.222.0.62 () {56 vars in 1114 bytes} [Mon Sep 5 12:10:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:38.150141 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0de4e-b698-4c83-9daa-199102301a32 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:38.262369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0de4e-b698-4c83-9daa-199102301a32 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:38.298402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb126aa0-4f1f-4d73-99ec-4c66dc60a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:38.299006 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb126aa0-4f1f-4d73-99ec-4c66dc60a7bd None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:38.299283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb126aa0-4f1f-4d73-99ec-4c66dc60a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:38.299572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb126aa0-4f1f-4d73-99ec-4c66dc60a7bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:38.300250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb126aa0-4f1f-4d73-99ec-4c66dc60a7bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:38.301601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb126aa0-4f1f-4d73-99ec-4c66dc60a7bd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:38.311051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:38.399276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 465/918] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:10:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:38.490349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf12a0-56ae-4777-804d-bd84b3386eac None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:38.490674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf12a0-56ae-4777-804d-bd84b3386eac None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:38.490886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf12a0-56ae-4777-804d-bd84b3386eac None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:38.492567 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 454/919] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:38.498327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84be788-de02-4f2f-b2b5-1ce82d7f0a10 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:38.498874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84be788-de02-4f2f-b2b5-1ce82d7f0a10 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:38.499147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84be788-de02-4f2f-b2b5-1ce82d7f0a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:38.519462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a84be788-de02-4f2f-b2b5-1ce82d7f0a10 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:38.520756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a84be788-de02-4f2f-b2b5-1ce82d7f0a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:38.605022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 466/920] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:38.873892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf12a0-56ae-4777-804d-bd84b3386eac None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:38.922236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:38.983217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf12a0-56ae-4777-804d-bd84b3386eac None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:39.017392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f05701-76eb-492e-9e6c-b63d7279ee04 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.017392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f05701-76eb-492e-9e6c-b63d7279ee04 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.017392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f05701-76eb-492e-9e6c-b63d7279ee04 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.018642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 467/921] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:39.022507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526da16-1fe5-496e-ba58-30646d9314cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:39.022507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526da16-1fe5-496e-ba58-30646d9314cd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.022507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526da16-1fe5-496e-ba58-30646d9314cd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.022507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526da16-1fe5-496e-ba58-30646d9314cd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.023143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526da16-1fe5-496e-ba58-30646d9314cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:39.023143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526da16-1fe5-496e-ba58-30646d9314cd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:39.024576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f3ef9-238b-40a0-b0dd-12ea7cc9dc16 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.025059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f3ef9-238b-40a0-b0dd-12ea7cc9dc16 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.025593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f3ef9-238b-40a0-b0dd-12ea7cc9dc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-c526da16-1fe5-496e-ba58-30646d9314cd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:39.032574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:39.033525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:39.034406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 455/922] 10.222.0.62 () {56 vars in 1132 bytes} [Mon Sep 5 12:10:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:39.039803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526da16-1fe5-496e-ba58-30646d9314cd None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:39.049596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1d9f3ef9-238b-40a0-b0dd-12ea7cc9dc16 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:39.051751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1d9f3ef9-238b-40a0-b0dd-12ea7cc9dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:39.143872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 468/923] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:39.174243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526da16-1fe5-496e-ba58-30646d9314cd None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:39.181551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:39.186063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1233a29-3a1f-4dc7-bffc-40ea3d565f10 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.186573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1233a29-3a1f-4dc7-bffc-40ea3d565f10 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.187033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1233a29-3a1f-4dc7-bffc-40ea3d565f10 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.189415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 469/924] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:10:39.197096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c410c-19df-4617-9e89-30860c87db89 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.197381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c410c-19df-4617-9e89-30860c87db89 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.197653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c410c-19df-4617-9e89-30860c87db89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.216952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25b7fa-209d-46e5-a379-10c79cae9a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:39.217915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25b7fa-209d-46e5-a379-10c79cae9a5d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.218410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25b7fa-209d-46e5-a379-10c79cae9a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.218901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25b7fa-209d-46e5-a379-10c79cae9a5d None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.219281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f16c410c-19df-4617-9e89-30860c87db89 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:39.219956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25b7fa-209d-46e5-a379-10c79cae9a5d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:39.220450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25b7fa-209d-46e5-a379-10c79cae9a5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:39.220569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f16c410c-19df-4617-9e89-30860c87db89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:39.222125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25b7fa-209d-46e5-a379-10c79cae9a5d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:39.227562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 456/925] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:39.236442 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25b7fa-209d-46e5-a379-10c79cae9a5d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:39.280710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:39.307679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 470/926] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:39.351120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25b7fa-209d-46e5-a379-10c79cae9a5d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:39.386710 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba0d21-99a5-405d-946d-f9eda3a0bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:39.387368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba0d21-99a5-405d-946d-f9eda3a0bc30 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.387667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba0d21-99a5-405d-946d-f9eda3a0bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.387938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba0d21-99a5-405d-946d-f9eda3a0bc30 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.388610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba0d21-99a5-405d-946d-f9eda3a0bc30 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:39.388909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba0d21-99a5-405d-946d-f9eda3a0bc30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:39.390273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba0d21-99a5-405d-946d-f9eda3a0bc30 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:39.405750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 457/927] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:39.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c410c-19df-4617-9e89-30860c87db89 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:39.413248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba0d21-99a5-405d-946d-f9eda3a0bc30 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:39.455364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:39.524802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba0d21-99a5-405d-946d-f9eda3a0bc30 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:39.532135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c410c-19df-4617-9e89-30860c87db89 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:39.560059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed7438-4e72-4fab-a471-71d404929cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:39.560838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed7438-4e72-4fab-a471-71d404929cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.561210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed7438-4e72-4fab-a471-71d404929cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.561577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed7438-4e72-4fab-a471-71d404929cd2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.576091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed7438-4e72-4fab-a471-71d404929cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:39.577292 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67955ea9-c644-46a3-adda-73eda9036292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:39.577913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67955ea9-c644-46a3-adda-73eda9036292 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.578213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67955ea9-c644-46a3-adda-73eda9036292 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.578489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67955ea9-c644-46a3-adda-73eda9036292 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.579155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67955ea9-c644-46a3-adda-73eda9036292 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:39.580714 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67955ea9-c644-46a3-adda-73eda9036292 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-08ed7438-4e72-4fab-a471-71d404929cd2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:39.581793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:39.582848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:10:39.583725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:39.584367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 458/928] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Sep 5 12:10:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:39.589369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed7438-4e72-4fab-a471-71d404929cd2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:39.668070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 471/929] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:10:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:39.676824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67955ea9-c644-46a3-adda-73eda9036292 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:39.684201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:39.692945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed7438-4e72-4fab-a471-71d404929cd2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:39.743051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ea3af-e6d9-4be2-b622-930908b91870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:39.743706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea3af-e6d9-4be2-b622-930908b91870 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.743985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea3af-e6d9-4be2-b622-930908b91870 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.744257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea3af-e6d9-4be2-b622-930908b91870 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.744933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ea3af-e6d9-4be2-b622-930908b91870 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:39.745245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ea3af-e6d9-4be2-b622-930908b91870 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:39.746642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ea3af-e6d9-4be2-b622-930908b91870 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:39.756163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:39.756496 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 459/930] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Sep 5 12:10:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:39.761847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ea3af-e6d9-4be2-b622-930908b91870 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:39.846344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67955ea9-c644-46a3-adda-73eda9036292 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:39.887837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faf402-a468-4acc-afd3-dc9de2aa91f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:39.888043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ea3af-e6d9-4be2-b622-930908b91870 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:39.889337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faf402-a468-4acc-afd3-dc9de2aa91f0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.889624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faf402-a468-4acc-afd3-dc9de2aa91f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.889898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faf402-a468-4acc-afd3-dc9de2aa91f0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.891047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faf402-a468-4acc-afd3-dc9de2aa91f0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:39.892310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faf402-a468-4acc-afd3-dc9de2aa91f0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-59faf402-a468-4acc-afd3-dc9de2aa91f0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:39.903900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:39.904944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:39.905803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 472/931] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:39.914228 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faf402-a468-4acc-afd3-dc9de2aa91f0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:39.938417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44da9a-5b13-4722-b1de-54fc6bba1182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:39.938949 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44da9a-5b13-4722-b1de-54fc6bba1182 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:39.939227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44da9a-5b13-4722-b1de-54fc6bba1182 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:39.939496 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44da9a-5b13-4722-b1de-54fc6bba1182 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:39.951243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44da9a-5b13-4722-b1de-54fc6bba1182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:39.952740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44da9a-5b13-4722-b1de-54fc6bba1182 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-2f44da9a-5b13-4722-b1de-54fc6bba1182 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:39.968029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:39.969077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:39.970031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:39.970031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:39.970031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:39.970031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 460/932] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:39.974669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44da9a-5b13-4722-b1de-54fc6bba1182 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:39.978952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:40.034116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faf402-a468-4acc-afd3-dc9de2aa91f0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:40.036437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:40.074175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0627c-ba5b-416f-a1e9-080db24c1c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:40.075032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0627c-ba5b-416f-a1e9-080db24c1c30 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.075481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0627c-ba5b-416f-a1e9-080db24c1c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.075915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0627c-ba5b-416f-a1e9-080db24c1c30 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.076829 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0627c-ba5b-416f-a1e9-080db24c1c30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:40.077306 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0627c-ba5b-416f-a1e9-080db24c1c30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:40.079066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0627c-ba5b-416f-a1e9-080db24c1c30 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:40.089514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44da9a-5b13-4722-b1de-54fc6bba1182 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:40.089911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 473/933] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Sep 5 12:10:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:40.091659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0627c-ba5b-416f-a1e9-080db24c1c30 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:40.124407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3deaeb-e5f4-42fc-912b-e0159e925617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:40.125012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3deaeb-e5f4-42fc-912b-e0159e925617 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.125883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3deaeb-e5f4-42fc-912b-e0159e925617 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.126156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3deaeb-e5f4-42fc-912b-e0159e925617 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.126794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3deaeb-e5f4-42fc-912b-e0159e925617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:40.127090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3deaeb-e5f4-42fc-912b-e0159e925617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:40.128801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3deaeb-e5f4-42fc-912b-e0159e925617 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:40.142181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 461/934] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:40.147162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3deaeb-e5f4-42fc-912b-e0159e925617 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:40.211465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0627c-ba5b-416f-a1e9-080db24c1c30 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:40.218865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:40.259099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3deaeb-e5f4-42fc-912b-e0159e925617 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:40.264087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876376bd-71ef-41b0-91a3-218b9bed6e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:40.264948 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876376bd-71ef-41b0-91a3-218b9bed6e50 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.265419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876376bd-71ef-41b0-91a3-218b9bed6e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.266406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876376bd-71ef-41b0-91a3-218b9bed6e50 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.267794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876376bd-71ef-41b0-91a3-218b9bed6e50 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:40.268303 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876376bd-71ef-41b0-91a3-218b9bed6e50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:40.270349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876376bd-71ef-41b0-91a3-218b9bed6e50 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:40.283177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 474/935] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Sep 5 12:10:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:40.284394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876376bd-71ef-41b0-91a3-218b9bed6e50 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:40.307937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b952b-5fce-4e8c-bb66-efd8a567956d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:40.308630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b952b-5fce-4e8c-bb66-efd8a567956d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.308912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b952b-5fce-4e8c-bb66-efd8a567956d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.309190 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b952b-5fce-4e8c-bb66-efd8a567956d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.316691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b952b-5fce-4e8c-bb66-efd8a567956d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:40.318394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b952b-5fce-4e8c-bb66-efd8a567956d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:40.324664 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-166b952b-5fce-4e8c-bb66-efd8a567956d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:40.331574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:40.332504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:40.333451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:40.333451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:40.333451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:40.333451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:40.333451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:40.333451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:40.333451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:40.333451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:40.333451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:40.333451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:40.337093 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 462/936] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Sep 5 12:10:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:40.337914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b952b-5fce-4e8c-bb66-efd8a567956d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:40.420695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876376bd-71ef-41b0-91a3-218b9bed6e50 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:40.446560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b952b-5fce-4e8c-bb66-efd8a567956d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:40.458673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d7404-dec0-4b34-b2e4-f400a2a2667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:40.459394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d7404-dec0-4b34-b2e4-f400a2a2667a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.459707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d7404-dec0-4b34-b2e4-f400a2a2667a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.459979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d7404-dec0-4b34-b2e4-f400a2a2667a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.460676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d7404-dec0-4b34-b2e4-f400a2a2667a None admin] RBAC: Adding query filter params (interface=public, service_id=e0b04830258a47249c3e79931626112d, region_id=RegionOne) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:40.460981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d7404-dec0-4b34-b2e4-f400a2a2667a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:40.462391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d7404-dec0-4b34-b2e4-f400a2a2667a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:40.471905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:40.472544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 475/937] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Sep 5 12:10:40 2022] GET /identity/v3/endpoints?service_id=e0b04830258a47249c3e79931626112d&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:40.473801 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16489621-9a5b-48bc-a407-732b058d7614 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.474086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16489621-9a5b-48bc-a407-732b058d7614 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.474361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16489621-9a5b-48bc-a407-732b058d7614 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.479573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 476/938] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:40.479573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16489621-9a5b-48bc-a407-732b058d7614 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:40.487518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03dade-4991-441e-94f5-5fafbdead417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:40.488174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03dade-4991-441e-94f5-5fafbdead417 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.488475 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03dade-4991-441e-94f5-5fafbdead417 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.488754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03dade-4991-441e-94f5-5fafbdead417 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.489506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03dade-4991-441e-94f5-5fafbdead417 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:40.489815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03dade-4991-441e-94f5-5fafbdead417 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:40.491273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03dade-4991-441e-94f5-5fafbdead417 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:40.511861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 463/939] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Sep 5 12:10:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:40.512959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cf09f-fa41-4ef0-afe5-7d34d9932a8d None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.513245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cf09f-fa41-4ef0-afe5-7d34d9932a8d None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.513519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cf09f-fa41-4ef0-afe5-7d34d9932a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.532791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-343cf09f-fa41-4ef0-afe5-7d34d9932a8d None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:40.533985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-343cf09f-fa41-4ef0-afe5-7d34d9932a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:40.579051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:40.586547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16489621-9a5b-48bc-a407-732b058d7614 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:40.609912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 464/940] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:40.611082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cf09f-fa41-4ef0-afe5-7d34d9932a8d None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:40.631226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77a2b8-78a9-450b-8f77-61240616769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:40.632186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77a2b8-78a9-450b-8f77-61240616769e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.632727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77a2b8-78a9-450b-8f77-61240616769e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.633811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77a2b8-78a9-450b-8f77-61240616769e None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.634967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77a2b8-78a9-450b-8f77-61240616769e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:40.637357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77a2b8-78a9-450b-8f77-61240616769e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:40.656654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 477/941] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Sep 5 12:10:40 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:10:40.731810 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cf09f-fa41-4ef0-afe5-7d34d9932a8d None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:40.747232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:40.774671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67473024-36a2-4c33-9281-49d5ce395ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:40.775794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473024-36a2-4c33-9281-49d5ce395ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.776279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473024-36a2-4c33-9281-49d5ce395ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.776754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473024-36a2-4c33-9281-49d5ce395ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.789801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67473024-36a2-4c33-9281-49d5ce395ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:40.792455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67473024-36a2-4c33-9281-49d5ce395ad8 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-67473024-36a2-4c33-9281-49d5ce395ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:40.811246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:40.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:40.814009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:40.814009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:40.814009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:40.815712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 465/942] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:40.822160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77a2b8-78a9-450b-8f77-61240616769e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:40.879604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67473024-36a2-4c33-9281-49d5ce395ad8 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:40.944789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77a2b8-78a9-450b-8f77-61240616769e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:40.981908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c26d9-3410-46f7-b52b-bc07accff35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:40.982826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c26d9-3410-46f7-b52b-bc07accff35f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:40.983268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c26d9-3410-46f7-b52b-bc07accff35f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:40.983729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c26d9-3410-46f7-b52b-bc07accff35f None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:40.985060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c26d9-3410-46f7-b52b-bc07accff35f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:40.985595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c26d9-3410-46f7-b52b-bc07accff35f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:40.987237 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c26d9-3410-46f7-b52b-bc07accff35f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:41.000816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 478/943] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:41.006005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c26d9-3410-46f7-b52b-bc07accff35f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:41.013919 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67473024-36a2-4c33-9281-49d5ce395ad8 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:41.029513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:41.062033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:41.062543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:41.063371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:41.063679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:41.067050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:41.067050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:41.070861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WARNING keystone.server.flask.common [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 05 12:10:41.124195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.sql.core [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 12:10:41.124195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 05 12:10:41.124195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [parameters: {'id': '892d36c357524b7a970a5b3a563a2059', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 05 12:10:41.124195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81648) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 892d36c357524b7a970a5b3a563a2059.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 892d36c357524b7a970a5b3a563a2059. Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:10:41.140237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application result.read() Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 05 12:10:41.141183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application yield resource Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise self.value Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application session.commit() Sep 05 12:10:41.142224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self.session.flush() Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application compat.raise_( Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 12:10:41.143141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 12:10:41.144095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application result.read() Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:10:41.147233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [parameters: {'id': '892d36c357524b7a970a5b3a563a2059', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:41.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:41.152342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:41.154627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:41.154627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 05 12:10:41.154627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 05 12:10:41.154627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:41.154627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:41.154627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 05 12:10:41.154627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 05 12:10:41.154627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 892d36c357524b7a970a5b3a563a2059. Sep 05 12:10:41.154627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:41.154627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 466/944] 10.222.0.62 () {60 vars in 1140 bytes} [Mon Sep 5 12:10:40 2022] POST /identity/v3/projects => generated 262 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:41.158128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c26d9-3410-46f7-b52b-bc07accff35f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:41.158617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:41.178710 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:41.199070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce971f3e-6b2c-4347-93b2-52cc48d07c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:41.199961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce971f3e-6b2c-4347-93b2-52cc48d07c5f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:41.200410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce971f3e-6b2c-4347-93b2-52cc48d07c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:41.200867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce971f3e-6b2c-4347-93b2-52cc48d07c5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:41.210188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce971f3e-6b2c-4347-93b2-52cc48d07c5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:41.212123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce971f3e-6b2c-4347-93b2-52cc48d07c5f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-ce971f3e-6b2c-4347-93b2-52cc48d07c5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:41.228675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:41.229778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:41.230865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:41.230865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:41.230865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:41.230865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:41.230865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:41.230865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:41.230865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:41.230865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:41.233077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 479/945] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:41.238853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce971f3e-6b2c-4347-93b2-52cc48d07c5f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:41.273624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae591a-ea09-4eb3-a629-02d30a3178ed None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:41.316516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871c88f-d837-49f5-b955-4cf98ab5a45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:41.317154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871c88f-d837-49f5-b955-4cf98ab5a45c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:41.317435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871c88f-d837-49f5-b955-4cf98ab5a45c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:41.317706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871c88f-d837-49f5-b955-4cf98ab5a45c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:41.326746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871c88f-d837-49f5-b955-4cf98ab5a45c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:41.329357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871c88f-d837-49f5-b955-4cf98ab5a45c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:41.341494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-2871c88f-d837-49f5-b955-4cf98ab5a45c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:41.345809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:41.346704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:41.347637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:41.347637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:41.347637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:41.347637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:41.347637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:41.347637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:41.347637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:41.347637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:41.347637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 467/946] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:41.355891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871c88f-d837-49f5-b955-4cf98ab5a45c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:41.370760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce971f3e-6b2c-4347-93b2-52cc48d07c5f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:41.411272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3017912-ac15-4f54-891a-ce023033e093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:41.412503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3017912-ac15-4f54-891a-ce023033e093 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:41.412987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3017912-ac15-4f54-891a-ce023033e093 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:41.413749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3017912-ac15-4f54-891a-ce023033e093 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:41.415115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3017912-ac15-4f54-891a-ce023033e093 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:41.415858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3017912-ac15-4f54-891a-ce023033e093 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:41.418106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3017912-ac15-4f54-891a-ce023033e093 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:41.435245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 480/947] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:41.443829 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3017912-ac15-4f54-891a-ce023033e093 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:41.467887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:41.488717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871c88f-d837-49f5-b955-4cf98ab5a45c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:41.538305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a36766-9167-4d73-b45b-10b00d45bb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:41.539432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a36766-9167-4d73-b45b-10b00d45bb5c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:41.539903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a36766-9167-4d73-b45b-10b00d45bb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:41.540586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a36766-9167-4d73-b45b-10b00d45bb5c None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:41.541835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a36766-9167-4d73-b45b-10b00d45bb5c None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:41.542525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a36766-9167-4d73-b45b-10b00d45bb5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:41.544759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a36766-9167-4d73-b45b-10b00d45bb5c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:41.550638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3017912-ac15-4f54-891a-ce023033e093 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:41.580199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 468/948] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:41.596608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7343ae-c2b9-4586-b1fc-6cee2063f469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:41.597299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7343ae-c2b9-4586-b1fc-6cee2063f469 None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:41.597576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7343ae-c2b9-4586-b1fc-6cee2063f469 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:41.597850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7343ae-c2b9-4586-b1fc-6cee2063f469 None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/81b72e82b4a34ab6af29ef440d19f6aa/roles/8b2491af21004040878bb0b8acb0194d` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:41.600265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7343ae-c2b9-4586-b1fc-6cee2063f469 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=81b72e82b4a34ab6af29ef440d19f6aa, role_id=8b2491af21004040878bb0b8acb0194d)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:41.601816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7343ae-c2b9-4586-b1fc-6cee2063f469 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:41.679106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:41.685268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 481/949] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:10:41 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/81b72e82b4a34ab6af29ef440d19f6aa/roles/8b2491af21004040878bb0b8acb0194d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:10:41.697112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4d49c-beff-4ac4-927b-342ee8b525e4 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:41.697402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4d49c-beff-4ac4-927b-342ee8b525e4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:41.697680 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4d49c-beff-4ac4-927b-342ee8b525e4 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:41.699173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 469/950] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:41.706086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe83fe5-93c5-401b-bb42-5361f1812558 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:41.706697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe83fe5-93c5-401b-bb42-5361f1812558 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:41.707131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe83fe5-93c5-401b-bb42-5361f1812558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:41.728073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7fe83fe5-93c5-401b-bb42-5361f1812558 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:41.729521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7fe83fe5-93c5-401b-bb42-5361f1812558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:41.812402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 482/951] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:42.091060 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4d49c-beff-4ac4-927b-342ee8b525e4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:42.199336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4d49c-beff-4ac4-927b-342ee8b525e4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:42.235800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa514a43-a6e3-4f2d-9297-24517300e26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:42.236760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa514a43-a6e3-4f2d-9297-24517300e26c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:42.237230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa514a43-a6e3-4f2d-9297-24517300e26c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:42.237849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa514a43-a6e3-4f2d-9297-24517300e26c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:42.238620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa514a43-a6e3-4f2d-9297-24517300e26c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:42.240239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa514a43-a6e3-4f2d-9297-24517300e26c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-fa514a43-a6e3-4f2d-9297-24517300e26c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:42.249324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:42.251033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:10:42.251974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:42.254678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 470/952] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:42.260745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe83fe5-93c5-401b-bb42-5361f1812558 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:42.363236 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe83fe5-93c5-401b-bb42-5361f1812558 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:42.391966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:42.398971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64101ff-12d1-4c4d-8aff-bc4391ddff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:42.399739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64101ff-12d1-4c4d-8aff-bc4391ddff77 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:42.400242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64101ff-12d1-4c4d-8aff-bc4391ddff77 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:42.400606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64101ff-12d1-4c4d-8aff-bc4391ddff77 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:42.401329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64101ff-12d1-4c4d-8aff-bc4391ddff77 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:42.401723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64101ff-12d1-4c4d-8aff-bc4391ddff77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:42.403152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64101ff-12d1-4c4d-8aff-bc4391ddff77 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:42.408177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 483/953] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:42.413838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa514a43-a6e3-4f2d-9297-24517300e26c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:42.417787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:42.516451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa514a43-a6e3-4f2d-9297-24517300e26c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:42.554387 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74e8a3-79c2-45da-91aa-bbe655a58000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:42.555020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74e8a3-79c2-45da-91aa-bbe655a58000 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:42.555322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74e8a3-79c2-45da-91aa-bbe655a58000 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:42.555646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74e8a3-79c2-45da-91aa-bbe655a58000 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:42.556299 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74e8a3-79c2-45da-91aa-bbe655a58000 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:42.556602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74e8a3-79c2-45da-91aa-bbe655a58000 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:42.557986 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74e8a3-79c2-45da-91aa-bbe655a58000 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:42.563556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 471/954] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:42.690318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a139c6b-6b3c-4086-8079-338d41220fd6 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:42.690519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a139c6b-6b3c-4086-8079-338d41220fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:42.690770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a139c6b-6b3c-4086-8079-338d41220fd6 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:42.692647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 484/955] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:42.698297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d7b14-9fb6-4579-a42f-ec97fb5daa94 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:42.698586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d7b14-9fb6-4579-a42f-ec97fb5daa94 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:42.698867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d7b14-9fb6-4579-a42f-ec97fb5daa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:42.718347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-006d7b14-9fb6-4579-a42f-ec97fb5daa94 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:42.720433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-006d7b14-9fb6-4579-a42f-ec97fb5daa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:42.786817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:42.792803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 472/956] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:43.064130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a139c6b-6b3c-4086-8079-338d41220fd6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:43.171293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a139c6b-6b3c-4086-8079-338d41220fd6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:43.207461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e81ca-b29f-4186-9000-8bca3772a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:43.208126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e81ca-b29f-4186-9000-8bca3772a240 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:43.208403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e81ca-b29f-4186-9000-8bca3772a240 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:43.208798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e81ca-b29f-4186-9000-8bca3772a240 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:43.209389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e81ca-b29f-4186-9000-8bca3772a240 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:43.210742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e81ca-b29f-4186-9000-8bca3772a240 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-ef4e81ca-b29f-4186-9000-8bca3772a240 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:43.220137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:43.221189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:43.222814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 485/957] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:43.227420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d7b14-9fb6-4579-a42f-ec97fb5daa94 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:43.342535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d7b14-9fb6-4579-a42f-ec97fb5daa94 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:43.379020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cc71b-21dd-40c2-8938-3a3a949b44e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:43.380060 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc71b-21dd-40c2-8938-3a3a949b44e3 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:43.380537 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc71b-21dd-40c2-8938-3a3a949b44e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:43.381036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc71b-21dd-40c2-8938-3a3a949b44e3 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:43.381941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cc71b-21dd-40c2-8938-3a3a949b44e3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:43.382456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cc71b-21dd-40c2-8938-3a3a949b44e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:43.384134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cc71b-21dd-40c2-8938-3a3a949b44e3 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:43.389750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 473/958] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Sep 5 12:10:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:43.396602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e81ca-b29f-4186-9000-8bca3772a240 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:43.456767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:43.504063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e81ca-b29f-4186-9000-8bca3772a240 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:43.539144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc128b-4c37-4b99-a534-d3c0e0171057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:43.540127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc128b-4c37-4b99-a534-d3c0e0171057 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:43.540616 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc128b-4c37-4b99-a534-d3c0e0171057 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:43.541101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc128b-4c37-4b99-a534-d3c0e0171057 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:43.541933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc128b-4c37-4b99-a534-d3c0e0171057 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:43.542425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc128b-4c37-4b99-a534-d3c0e0171057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:43.543997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc128b-4c37-4b99-a534-d3c0e0171057 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:43.550194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 486/959] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Sep 5 12:10:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:43.556680 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cc71b-21dd-40c2-8938-3a3a949b44e3 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:43.625733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:43.653981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6527-d708-4fb7-8904-829aedc492b1 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:43.654609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6527-d708-4fb7-8904-829aedc492b1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:43.655089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6527-d708-4fb7-8904-829aedc492b1 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:43.661342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 487/960] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:10:43.663016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079721-c3be-4cc2-8943-b6c349cff125 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:43.663516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079721-c3be-4cc2-8943-b6c349cff125 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:43.664027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079721-c3be-4cc2-8943-b6c349cff125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:43.666131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cc71b-21dd-40c2-8938-3a3a949b44e3 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:43.685752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7a079721-c3be-4cc2-8943-b6c349cff125 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:43.687563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7a079721-c3be-4cc2-8943-b6c349cff125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:43.709707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21462b-67f8-4de2-b4b0-25a692b187e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:43.710315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21462b-67f8-4de2-b4b0-25a692b187e7 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:43.710594 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21462b-67f8-4de2-b4b0-25a692b187e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:43.710866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21462b-67f8-4de2-b4b0-25a692b187e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:43.711549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21462b-67f8-4de2-b4b0-25a692b187e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:43.713399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21462b-67f8-4de2-b4b0-25a692b187e7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:43.762727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 488/961] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:43.763998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c658651-b696-46d2-87a0-80a27f2749fe None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:43.764280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c658651-b696-46d2-87a0-80a27f2749fe None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:43.764548 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c658651-b696-46d2-87a0-80a27f2749fe None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:43.766432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 489/962] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:43.771217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab720c-4b64-44ee-a779-8ea34877cfc9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:43.771493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab720c-4b64-44ee-a779-8ea34877cfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:43.771891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab720c-4b64-44ee-a779-8ea34877cfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:43.789775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ecab720c-4b64-44ee-a779-8ea34877cfc9 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:43.790938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ecab720c-4b64-44ee-a779-8ea34877cfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:43.811972 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 474/963] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:10:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:43.850652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:43.882583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 490/964] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:44.056633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21462b-67f8-4de2-b4b0-25a692b187e7 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:44.148880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab720c-4b64-44ee-a779-8ea34877cfc9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:44.177406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21462b-67f8-4de2-b4b0-25a692b187e7 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:44.208295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:44.232914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1402c9-018c-45c0-b910-f7c1627132d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:44.233562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1402c9-018c-45c0-b910-f7c1627132d8 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:44.233837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1402c9-018c-45c0-b910-f7c1627132d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:44.234065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1402c9-018c-45c0-b910-f7c1627132d8 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:44.234738 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1402c9-018c-45c0-b910-f7c1627132d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:44.236255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1402c9-018c-45c0-b910-f7c1627132d8 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:44.265925 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab720c-4b64-44ee-a779-8ea34877cfc9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:44.283982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.sql.core [None req-9b1402c9-018c-45c0-b910-f7c1627132d8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 12:10:44.283982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 05 12:10:44.283982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [parameters: {'user_id': '0ca925b946214eaa8b5d681890b2241a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 05 12:10:44.283982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81648) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-9b1402c9-018c-45c0-b910-f7c1627132d8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application result.read() Sep 05 12:10:44.295335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:10:44.296378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application yield resource Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application next(self.gen) Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise self.value Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application session.commit() Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 12:10:44.297982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self.session.flush() Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application compat.raise_( Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 12:10:44.298903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 12:10:44.300313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application conn.query(q) Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application result.read() Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 12:10:44.301403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [parameters: {'user_id': '0ca925b946214eaa8b5d681890b2241a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:44.302488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 05 12:10:44.303529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 05 12:10:44.304808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 05 12:10:44.304808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 05 12:10:44.304808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 12:10:44.304808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 12:10:44.304808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 05 12:10:44.304808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:44.307052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 475/965] 10.222.0.62 () {60 vars in 1131 bytes} [Mon Sep 5 12:10:44 2022] POST /identity/v3/users => generated 160 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:44.312860 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1402c9-018c-45c0-b910-f7c1627132d8 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:44.321593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634a6e8-5b1c-4e65-9a2d-360a4b2d4e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:44.322323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634a6e8-5b1c-4e65-9a2d-360a4b2d4e62 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:44.322604 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634a6e8-5b1c-4e65-9a2d-360a4b2d4e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:44.324274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634a6e8-5b1c-4e65-9a2d-360a4b2d4e62 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:44.338475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634a6e8-5b1c-4e65-9a2d-360a4b2d4e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-8634a6e8-5b1c-4e65-9a2d-360a4b2d4e62 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:44.346896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:44.347986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:44.348888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 491/966] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Sep 5 12:10:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:44.357661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634a6e8-5b1c-4e65-9a2d-360a4b2d4e62 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:44.410722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:44.446046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1402c9-018c-45c0-b910-f7c1627132d8 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:44.473975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634a6e8-5b1c-4e65-9a2d-360a4b2d4e62 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:44.484552 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba73bc9-4480-463c-a310-02870ab18623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:44.486253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73bc9-4480-463c-a310-02870ab18623 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:44.486698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73bc9-4480-463c-a310-02870ab18623 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:44.487130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73bc9-4480-463c-a310-02870ab18623 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:44.498154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba73bc9-4480-463c-a310-02870ab18623 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:44.500202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba73bc9-4480-463c-a310-02870ab18623 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-2ba73bc9-4480-463c-a310-02870ab18623 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:44.513432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:44.514447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:44.515394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:44.515394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:44.515394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:44.515394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:44.515394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:44.515394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:44.515394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:44.515394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:44.515394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 12:10:44.515394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:44.516691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38478eb1-dd93-4e6a-84dc-94c074a7bc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:44.516691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38478eb1-dd93-4e6a-84dc-94c074a7bc88 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:44.516691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38478eb1-dd93-4e6a-84dc-94c074a7bc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:44.516691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38478eb1-dd93-4e6a-84dc-94c074a7bc88 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:44.516983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38478eb1-dd93-4e6a-84dc-94c074a7bc88 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:44.516983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38478eb1-dd93-4e6a-84dc-94c074a7bc88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:44.518782 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38478eb1-dd93-4e6a-84dc-94c074a7bc88 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:44.519494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 476/967] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:44.525455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba73bc9-4480-463c-a310-02870ab18623 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:44.539263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 492/968] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Sep 5 12:10:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:44.540890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38478eb1-dd93-4e6a-84dc-94c074a7bc88 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:44.575091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:44.638402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba73bc9-4480-463c-a310-02870ab18623 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:44.645057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:44.650797 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38478eb1-dd93-4e6a-84dc-94c074a7bc88 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:44.674252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce609b6-cf2f-4525-bf10-cf7f196ac345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:44.674787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce609b6-cf2f-4525-bf10-cf7f196ac345 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:44.675069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce609b6-cf2f-4525-bf10-cf7f196ac345 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:44.675350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce609b6-cf2f-4525-bf10-cf7f196ac345 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:44.676074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce609b6-cf2f-4525-bf10-cf7f196ac345 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:44.676379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce609b6-cf2f-4525-bf10-cf7f196ac345 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:44.677916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce609b6-cf2f-4525-bf10-cf7f196ac345 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:44.686114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f597b9-9744-471c-871d-ef650130198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:44.686965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f597b9-9744-471c-871d-ef650130198c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:44.687435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f597b9-9744-471c-871d-ef650130198c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:44.687949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f597b9-9744-471c-871d-ef650130198c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:44.699724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f597b9-9744-471c-871d-ef650130198c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:44.701507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f597b9-9744-471c-871d-ef650130198c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:44.716162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 477/969] 10.222.0.62 () {56 vars in 1101 bytes} [Mon Sep 5 12:10:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-04f597b9-9744-471c-871d-ef650130198c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:44.717051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:44.718178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:44.719942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:44.719942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:44.719942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:44.721319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03444104-0365-40b9-b1e3-01519e418ccf None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:44.721794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03444104-0365-40b9-b1e3-01519e418ccf None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:44.722241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03444104-0365-40b9-b1e3-01519e418ccf None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:44.723773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 493/970] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:44.724634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 478/971] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:44.724763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f597b9-9744-471c-871d-ef650130198c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:44.730662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a952d2-9a59-4bbf-bced-9d2a7042b3a0 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:44.730951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a952d2-9a59-4bbf-bced-9d2a7042b3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:44.731216 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a952d2-9a59-4bbf-bced-9d2a7042b3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:44.751006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-89a952d2-9a59-4bbf-bced-9d2a7042b3a0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:44.752700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-89a952d2-9a59-4bbf-bced-9d2a7042b3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:44.839921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 479/972] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:44.858770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f597b9-9744-471c-871d-ef650130198c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:44.895348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43327ac-a74c-41b6-a665-3d9ccba4d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:44.895985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43327ac-a74c-41b6-a665-3d9ccba4d198 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:44.896261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43327ac-a74c-41b6-a665-3d9ccba4d198 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:44.896530 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43327ac-a74c-41b6-a665-3d9ccba4d198 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:44.897169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43327ac-a74c-41b6-a665-3d9ccba4d198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:44.897466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43327ac-a74c-41b6-a665-3d9ccba4d198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:44.898820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43327ac-a74c-41b6-a665-3d9ccba4d198 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:44.911444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 494/973] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:44.916753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a952d2-9a59-4bbf-bced-9d2a7042b3a0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:44.949835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:45.029972 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a952d2-9a59-4bbf-bced-9d2a7042b3a0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:45.073042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d219e-07ef-44dc-849c-34c3ee449b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:45.073754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d219e-07ef-44dc-849c-34c3ee449b05 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:45.074102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d219e-07ef-44dc-849c-34c3ee449b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:45.074442 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d219e-07ef-44dc-849c-34c3ee449b05 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:45.082077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d219e-07ef-44dc-849c-34c3ee449b05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:45.083644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d219e-07ef-44dc-849c-34c3ee449b05 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-c30d219e-07ef-44dc-849c-34c3ee449b05 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:45.095339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:45.096499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:45.097777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:45.097777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:45.097777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:45.097777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:45.097777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:45.097777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:45.097777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:45.097777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:45.097777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 12:10:45.097777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:45.098907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 480/974] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Sep 5 12:10:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:45.104471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43327ac-a74c-41b6-a665-3d9ccba4d198 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:45.111691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d219e-07ef-44dc-849c-34c3ee449b05 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:45.118140 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:45.204973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:45.209468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43327ac-a74c-41b6-a665-3d9ccba4d198 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:45.217542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d219e-07ef-44dc-849c-34c3ee449b05 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:45.245212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc4278-e619-4f12-b2f0-1541e70a623b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:45.245875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc4278-e619-4f12-b2f0-1541e70a623b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:45.246156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc4278-e619-4f12-b2f0-1541e70a623b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:45.246424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc4278-e619-4f12-b2f0-1541e70a623b None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:45.247147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc4278-e619-4f12-b2f0-1541e70a623b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:45.247446 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc4278-e619-4f12-b2f0-1541e70a623b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:45.248900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc4278-e619-4f12-b2f0-1541e70a623b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:45.255370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6d574-1f52-4794-b1ae-ee2ffa83bc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:45.256076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6d574-1f52-4794-b1ae-ee2ffa83bc25 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:45.256333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6d574-1f52-4794-b1ae-ee2ffa83bc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:45.256543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6d574-1f52-4794-b1ae-ee2ffa83bc25 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:45.257184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6d574-1f52-4794-b1ae-ee2ffa83bc25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:45.259168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6d574-1f52-4794-b1ae-ee2ffa83bc25 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:45.268195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 495/975] 10.222.0.62 () {56 vars in 1149 bytes} [Mon Sep 5 12:10:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:45.273773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc4278-e619-4f12-b2f0-1541e70a623b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:45.338131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:45.357820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 481/976] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:10:45 2022] POST /identity/v3/services => generated 242 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:10:45.373198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc4278-e619-4f12-b2f0-1541e70a623b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:45.420427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e784267-f7cc-4c98-823e-8fa07834541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:45.421093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e784267-f7cc-4c98-823e-8fa07834541c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:45.421334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e784267-f7cc-4c98-823e-8fa07834541c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:45.421570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e784267-f7cc-4c98-823e-8fa07834541c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:45.496695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e784267-f7cc-4c98-823e-8fa07834541c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:45.498047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e784267-f7cc-4c98-823e-8fa07834541c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:45.559458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:45.603383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 496/977] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:45 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:10:45.605051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6d574-1f52-4794-b1ae-ee2ffa83bc25 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:45.706335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6d574-1f52-4794-b1ae-ee2ffa83bc25 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:45.741292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79351540-e9cd-403b-a363-7340c6e7ee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:45.742004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79351540-e9cd-403b-a363-7340c6e7ee05 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:45.742344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79351540-e9cd-403b-a363-7340c6e7ee05 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:45.742677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79351540-e9cd-403b-a363-7340c6e7ee05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:45.753188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79351540-e9cd-403b-a363-7340c6e7ee05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:45.754628 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79351540-e9cd-403b-a363-7340c6e7ee05 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:45.762538 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-79351540-e9cd-403b-a363-7340c6e7ee05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:45.769576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:45.770886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:45.776694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:45.776694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:45.776694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:45.776694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 482/978] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:45.778514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e784267-f7cc-4c98-823e-8fa07834541c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:45.812123 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f431a4-1647-401a-86cb-4743086fbce7 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:45.812505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f431a4-1647-401a-86cb-4743086fbce7 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:45.812793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f431a4-1647-401a-86cb-4743086fbce7 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:45.814223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 483/979] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:10:45.819345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a959d6c-9ff1-483d-9f5a-35ca13f7a4bb None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:45.822430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a959d6c-9ff1-483d-9f5a-35ca13f7a4bb None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:45.822430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a959d6c-9ff1-483d-9f5a-35ca13f7a4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:45.838494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6a959d6c-9ff1-483d-9f5a-35ca13f7a4bb None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:45.839782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6a959d6c-9ff1-483d-9f5a-35ca13f7a4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:45.894520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e784267-f7cc-4c98-823e-8fa07834541c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:45.913197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 484/980] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:45.934264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a92ace-f583-480e-849e-65633fba4798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:45.934983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92ace-f583-480e-849e-65633fba4798 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:45.935353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92ace-f583-480e-849e-65633fba4798 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:45.935751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92ace-f583-480e-849e-65633fba4798 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:45.936481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a92ace-f583-480e-849e-65633fba4798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:45.936875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a92ace-f583-480e-849e-65633fba4798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:45.938368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a92ace-f583-480e-849e-65633fba4798 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:45.954099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 497/981] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:45.955754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a959d6c-9ff1-483d-9f5a-35ca13f7a4bb None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:46.061264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a959d6c-9ff1-483d-9f5a-35ca13f7a4bb None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:46.093388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:46.099708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebc3e1-e52c-4065-8207-38389449c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:46.100336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebc3e1-e52c-4065-8207-38389449c9fc None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:46.100739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebc3e1-e52c-4065-8207-38389449c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:46.100912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebc3e1-e52c-4065-8207-38389449c9fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:46.107362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebc3e1-e52c-4065-8207-38389449c9fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:46.108839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebc3e1-e52c-4065-8207-38389449c9fc None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-65ebc3e1-e52c-4065-8207-38389449c9fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:46.119625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:46.120507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:46.126319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:46.126319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:46.126319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:46.126319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:46.126319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:46.126319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:46.126319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:46.126319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:46.126319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 485/982] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:46.128570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a92ace-f583-480e-849e-65633fba4798 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:46.156047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:46.182001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebc3e1-e52c-4065-8207-38389449c9fc None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:46.237350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a92ace-f583-480e-849e-65633fba4798 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:46.273976 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651cb46-a97a-4c26-bc30-4f5e487f92ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:46.274726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651cb46-a97a-4c26-bc30-4f5e487f92ba None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:46.275120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651cb46-a97a-4c26-bc30-4f5e487f92ba None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:46.275396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651cb46-a97a-4c26-bc30-4f5e487f92ba None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:46.276262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651cb46-a97a-4c26-bc30-4f5e487f92ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:46.276569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651cb46-a97a-4c26-bc30-4f5e487f92ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:46.278558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651cb46-a97a-4c26-bc30-4f5e487f92ba None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:46.285874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebc3e1-e52c-4065-8207-38389449c9fc None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:46.293108 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 498/983] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:46.298601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651cb46-a97a-4c26-bc30-4f5e487f92ba None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:46.324427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6660b04-48c1-4e3a-a3cd-3a508fb43160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:46.324983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6660b04-48c1-4e3a-a3cd-3a508fb43160 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:46.325252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6660b04-48c1-4e3a-a3cd-3a508fb43160 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:46.325528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6660b04-48c1-4e3a-a3cd-3a508fb43160 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:46.326117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6660b04-48c1-4e3a-a3cd-3a508fb43160 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:46.327489 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6660b04-48c1-4e3a-a3cd-3a508fb43160 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-d6660b04-48c1-4e3a-a3cd-3a508fb43160 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:46.339127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:46.340423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:46.341320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 486/984] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:46.346840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6660b04-48c1-4e3a-a3cd-3a508fb43160 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:46.388857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:46.404970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651cb46-a97a-4c26-bc30-4f5e487f92ba None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:46.412814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:46.442479 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e5c8d-7b82-4c6c-85e1-305304d38a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:46.443102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5c8d-7b82-4c6c-85e1-305304d38a7d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:46.443382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5c8d-7b82-4c6c-85e1-305304d38a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:46.443683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5c8d-7b82-4c6c-85e1-305304d38a7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:46.444441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e5c8d-7b82-4c6c-85e1-305304d38a7d None admin] RBAC: Adding query filter params (role.id=8b2491af21004040878bb0b8acb0194d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=81b72e82b4a34ab6af29ef440d19f6aa) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:46.444746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e5c8d-7b82-4c6c-85e1-305304d38a7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:46.446214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e5c8d-7b82-4c6c-85e1-305304d38a7d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:46.453150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 499/985] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:46 2022] GET /identity/v3/role_assignments?user.id=81b72e82b4a34ab6af29ef440d19f6aa&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=8b2491af21004040878bb0b8acb0194d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:46.461098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6660b04-48c1-4e3a-a3cd-3a508fb43160 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:46.501638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd88e5-f429-4dbd-a728-8ca05f4e527b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:46.502429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd88e5-f429-4dbd-a728-8ca05f4e527b None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:46.502811 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd88e5-f429-4dbd-a728-8ca05f4e527b None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:46.503186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd88e5-f429-4dbd-a728-8ca05f4e527b None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:46.503987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd88e5-f429-4dbd-a728-8ca05f4e527b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:46.504433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd88e5-f429-4dbd-a728-8ca05f4e527b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:46.505969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd88e5-f429-4dbd-a728-8ca05f4e527b None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:46.517343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 487/986] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Sep 5 12:10:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:46.518516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e5c8d-7b82-4c6c-85e1-305304d38a7d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:46.629441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e5c8d-7b82-4c6c-85e1-305304d38a7d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:46.664320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12e6e6-727b-4355-8abe-1a265a7f9a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:46.664952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12e6e6-727b-4355-8abe-1a265a7f9a7e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:46.665232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12e6e6-727b-4355-8abe-1a265a7f9a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:46.665505 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12e6e6-727b-4355-8abe-1a265a7f9a7e None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:46.666135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12e6e6-727b-4355-8abe-1a265a7f9a7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:46.666437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12e6e6-727b-4355-8abe-1a265a7f9a7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:46.667827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12e6e6-727b-4355-8abe-1a265a7f9a7e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:46.672819 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 500/987] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Sep 5 12:10:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:46.678755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd88e5-f429-4dbd-a728-8ca05f4e527b None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:46.780017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd88e5-f429-4dbd-a728-8ca05f4e527b None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:46.798177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:46.815716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721978ab-33b4-4044-9d11-67508d09e753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:46.816540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721978ab-33b4-4044-9d11-67508d09e753 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:46.816854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721978ab-33b4-4044-9d11-67508d09e753 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:46.817199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721978ab-33b4-4044-9d11-67508d09e753 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:46.817986 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721978ab-33b4-4044-9d11-67508d09e753 None admin] RBAC: Adding query filter params (interface=admin, service_id=e0b04830258a47249c3e79931626112d, region_id=RegionOne) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:46.818368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721978ab-33b4-4044-9d11-67508d09e753 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:46.819831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721978ab-33b4-4044-9d11-67508d09e753 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:46.829715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 488/988] 10.222.0.62 () {56 vars in 1249 bytes} [Mon Sep 5 12:10:46 2022] GET /identity/v3/endpoints?service_id=e0b04830258a47249c3e79931626112d&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:46.831109 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12e6e6-727b-4355-8abe-1a265a7f9a7e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:46.846508 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:46.932291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12e6e6-727b-4355-8abe-1a265a7f9a7e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:46.966625 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66ef11-8cb8-40de-b157-add8e6560808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:46.967485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66ef11-8cb8-40de-b157-add8e6560808 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:46.967785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66ef11-8cb8-40de-b157-add8e6560808 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:46.968052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66ef11-8cb8-40de-b157-add8e6560808 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:46.968649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66ef11-8cb8-40de-b157-add8e6560808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:46.970041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66ef11-8cb8-40de-b157-add8e6560808 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:46.976396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 501/989] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Sep 5 12:10:46 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:10:47.282502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1eee7-4126-4f6b-8478-642ec829b439 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:47.282862 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1eee7-4126-4f6b-8478-642ec829b439 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:47.283057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1eee7-4126-4f6b-8478-642ec829b439 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:47.284471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 489/990] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:47.289502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b7890-e4f3-4833-bfcf-147b8cffc5fc None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:47.289787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b7890-e4f3-4833-bfcf-147b8cffc5fc None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:47.290061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b7890-e4f3-4833-bfcf-147b8cffc5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:47.309179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-645b7890-e4f3-4833-bfcf-147b8cffc5fc None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:47.310367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-645b7890-e4f3-4833-bfcf-147b8cffc5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:47.378900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 502/991] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:47.646286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1eee7-4126-4f6b-8478-642ec829b439 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:47.746373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1eee7-4126-4f6b-8478-642ec829b439 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:47.779611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58fa2a-19bb-4b4d-90d1-b75cd7213daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:47.780233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58fa2a-19bb-4b4d-90d1-b75cd7213daf None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:47.780511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58fa2a-19bb-4b4d-90d1-b75cd7213daf None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:47.780780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58fa2a-19bb-4b4d-90d1-b75cd7213daf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:47.781348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58fa2a-19bb-4b4d-90d1-b75cd7213daf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:47.783269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58fa2a-19bb-4b4d-90d1-b75cd7213daf None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-4f58fa2a-19bb-4b4d-90d1-b75cd7213daf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:47.792112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:47.793127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:10:47.794023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:47.795894 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 490/992] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:47.799043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b7890-e4f3-4833-bfcf-147b8cffc5fc None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:47.807230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:47.898504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b7890-e4f3-4833-bfcf-147b8cffc5fc None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:47.933075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774cb7-830b-47af-b8ae-ac2d5e79bbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:47.933851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774cb7-830b-47af-b8ae-ac2d5e79bbdf None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:47.934226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774cb7-830b-47af-b8ae-ac2d5e79bbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:47.934598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774cb7-830b-47af-b8ae-ac2d5e79bbdf None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:47.935512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0774cb7-830b-47af-b8ae-ac2d5e79bbdf None admin] RBAC: Adding query filter params (name=compute) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:47.935800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0774cb7-830b-47af-b8ae-ac2d5e79bbdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:47.937231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0774cb7-830b-47af-b8ae-ac2d5e79bbdf None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:47.943023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 503/993] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:47.948468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58fa2a-19bb-4b4d-90d1-b75cd7213daf None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:47.996245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:48.058679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58fa2a-19bb-4b4d-90d1-b75cd7213daf None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:48.094681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2331-ac10-46bc-a436-323984e1af55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:48.095401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2331-ac10-46bc-a436-323984e1af55 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.095772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2331-ac10-46bc-a436-323984e1af55 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.096109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2331-ac10-46bc-a436-323984e1af55 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.096801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc2331-ac10-46bc-a436-323984e1af55 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:48.097168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc2331-ac10-46bc-a436-323984e1af55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:48.098605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc2331-ac10-46bc-a436-323984e1af55 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:48.103648 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 491/994] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:48.109252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774cb7-830b-47af-b8ae-ac2d5e79bbdf None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:48.178194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:48.208963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774cb7-830b-47af-b8ae-ac2d5e79bbdf None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:48.243563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b90054-0506-443e-8512-d6ca7cb001b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:48.244277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b90054-0506-443e-8512-d6ca7cb001b8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.244622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b90054-0506-443e-8512-d6ca7cb001b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.244959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b90054-0506-443e-8512-d6ca7cb001b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.245723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b90054-0506-443e-8512-d6ca7cb001b8 None admin] RBAC: Adding query filter params (interface=public, service_id=9474c4cfded241ee80062f278c89975a, region_id=RegionOne) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:48.246105 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b90054-0506-443e-8512-d6ca7cb001b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:48.247517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b90054-0506-443e-8512-d6ca7cb001b8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:48.253405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 504/995] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Sep 5 12:10:48 2022] GET /identity/v3/endpoints?service_id=9474c4cfded241ee80062f278c89975a&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:48.258770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2331-ac10-46bc-a436-323984e1af55 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:48.362590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2331-ac10-46bc-a436-323984e1af55 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:48.370859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399d732-812f-40cc-a213-1906a98576bb None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.371143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399d732-812f-40cc-a213-1906a98576bb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.371418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399d732-812f-40cc-a213-1906a98576bb None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.372910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 505/996] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:10:48.373501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:48.378028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ff7c0-58d2-4bd8-b9e4-84c3c7e8b42b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.378311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ff7c0-58d2-4bd8-b9e4-84c3c7e8b42b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.378583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ff7c0-58d2-4bd8-b9e4-84c3c7e8b42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.397096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b63ff7c0-58d2-4bd8-b9e4-84c3c7e8b42b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:48.398268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b63ff7c0-58d2-4bd8-b9e4-84c3c7e8b42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:48.401760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26198f-6089-4b22-bbda-a77b9f401f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:48.402649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26198f-6089-4b22-bbda-a77b9f401f18 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.403182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26198f-6089-4b22-bbda-a77b9f401f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.403660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26198f-6089-4b22-bbda-a77b9f401f18 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.404490 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26198f-6089-4b22-bbda-a77b9f401f18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:48.406470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26198f-6089-4b22-bbda-a77b9f401f18 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:48.415606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 492/997] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Sep 5 12:10:48 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:10:48.475981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 506/998] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:48.745917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26198f-6089-4b22-bbda-a77b9f401f18 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:48.846045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26198f-6089-4b22-bbda-a77b9f401f18 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:48.869876 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602762d-7990-4b78-83a0-18f12df8195f None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.870379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602762d-7990-4b78-83a0-18f12df8195f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.870501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602762d-7990-4b78-83a0-18f12df8195f None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.871917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 507/999] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:10:48.876702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158198a-5d73-4970-8118-cba54ded0c8e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.876981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158198a-5d73-4970-8118-cba54ded0c8e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.877253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158198a-5d73-4970-8118-cba54ded0c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.884610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7b1ba-84b9-4b8c-90fe-2f1fb149fe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:48.885243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7b1ba-84b9-4b8c-90fe-2f1fb149fe98 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:48.885519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7b1ba-84b9-4b8c-90fe-2f1fb149fe98 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:48.885791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7b1ba-84b9-4b8c-90fe-2f1fb149fe98 None admin] PATH_INFO: `/v3/services/image` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:48.886363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7b1ba-84b9-4b8c-90fe-2f1fb149fe98 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:48.887746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7b1ba-84b9-4b8c-90fe-2f1fb149fe98 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:48.894988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9158198a-5d73-4970-8118-cba54ded0c8e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:48.896194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9158198a-5d73-4970-8118-cba54ded0c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-aaf7b1ba-84b9-4b8c-90fe-2f1fb149fe98 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:48.896722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:48.897707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:10:48.898563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:48.900668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 493/1000] 10.222.0.62 () {56 vars in 1105 bytes} [Mon Sep 5 12:10:48 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:48.903482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7b1ba-84b9-4b8c-90fe-2f1fb149fe98 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:48.947840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:48.972520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:48.981314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 508/1001] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:49.019154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7b1ba-84b9-4b8c-90fe-2f1fb149fe98 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:49.057504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8d366-a22f-4f0b-89e6-75ea57abd51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:49.058120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8d366-a22f-4f0b-89e6-75ea57abd51d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:49.058402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8d366-a22f-4f0b-89e6-75ea57abd51d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:49.058678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8d366-a22f-4f0b-89e6-75ea57abd51d None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:49.059309 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8d366-a22f-4f0b-89e6-75ea57abd51d None admin] RBAC: Adding query filter params (name=image) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:49.059636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8d366-a22f-4f0b-89e6-75ea57abd51d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:49.061026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8d366-a22f-4f0b-89e6-75ea57abd51d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:49.072876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 494/1002] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:49.074325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158198a-5d73-4970-8118-cba54ded0c8e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:49.176362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158198a-5d73-4970-8118-cba54ded0c8e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:49.210104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25451b35-8a71-4bc1-9e2d-f02f6adc8128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:49.210704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25451b35-8a71-4bc1-9e2d-f02f6adc8128 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:49.210982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25451b35-8a71-4bc1-9e2d-f02f6adc8128 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:49.211259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25451b35-8a71-4bc1-9e2d-f02f6adc8128 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:49.211914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25451b35-8a71-4bc1-9e2d-f02f6adc8128 None admin] RBAC: Adding query filter params (type=image) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:49.212217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25451b35-8a71-4bc1-9e2d-f02f6adc8128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:49.213567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25451b35-8a71-4bc1-9e2d-f02f6adc8128 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:49.218437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 509/1003] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:49.245527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8d366-a22f-4f0b-89e6-75ea57abd51d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:49.354235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8d366-a22f-4f0b-89e6-75ea57abd51d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:49.355832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:49.393757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921895d2-c047-40df-b1e2-f30757cbe981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:49.394385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921895d2-c047-40df-b1e2-f30757cbe981 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:49.394659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921895d2-c047-40df-b1e2-f30757cbe981 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:49.394926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921895d2-c047-40df-b1e2-f30757cbe981 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:49.395500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921895d2-c047-40df-b1e2-f30757cbe981 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:49.396906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921895d2-c047-40df-b1e2-f30757cbe981 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-921895d2-c047-40df-b1e2-f30757cbe981 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:49.405898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:49.406945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:49.407955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 495/1004] 10.222.0.62 () {56 vars in 1126 bytes} [Mon Sep 5 12:10:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:49.413630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25451b35-8a71-4bc1-9e2d-f02f6adc8128 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:49.476056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:49.521737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25451b35-8a71-4bc1-9e2d-f02f6adc8128 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:49.555762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575d034-2da5-46cb-85da-cfcce8a96ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:49.556375 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575d034-2da5-46cb-85da-cfcce8a96ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:49.556656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575d034-2da5-46cb-85da-cfcce8a96ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:49.556931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575d034-2da5-46cb-85da-cfcce8a96ae6 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:49.557588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575d034-2da5-46cb-85da-cfcce8a96ae6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:49.557889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575d034-2da5-46cb-85da-cfcce8a96ae6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:49.559253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575d034-2da5-46cb-85da-cfcce8a96ae6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:49.569388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 510/1005] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Sep 5 12:10:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:49.570614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921895d2-c047-40df-b1e2-f30757cbe981 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:49.676465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921895d2-c047-40df-b1e2-f30757cbe981 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:49.710375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a68491-361d-423d-96d1-7a8e2eb4bbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:49.710982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a68491-361d-423d-96d1-7a8e2eb4bbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:49.711260 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a68491-361d-423d-96d1-7a8e2eb4bbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:49.711530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a68491-361d-423d-96d1-7a8e2eb4bbc8 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:49.712177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a68491-361d-423d-96d1-7a8e2eb4bbc8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:49.712478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a68491-361d-423d-96d1-7a8e2eb4bbc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:49.713834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a68491-361d-423d-96d1-7a8e2eb4bbc8 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:49.723627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 496/1006] 10.222.0.62 () {56 vars in 1122 bytes} [Mon Sep 5 12:10:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:49.725121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575d034-2da5-46cb-85da-cfcce8a96ae6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:49.827098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575d034-2da5-46cb-85da-cfcce8a96ae6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:49.850232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:49.862656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884b19e-95e9-483a-aa72-5d7fb6000bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:49.863261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884b19e-95e9-483a-aa72-5d7fb6000bab None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:49.863549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884b19e-95e9-483a-aa72-5d7fb6000bab None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:49.867519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884b19e-95e9-483a-aa72-5d7fb6000bab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:49.867519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884b19e-95e9-483a-aa72-5d7fb6000bab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:49.867519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884b19e-95e9-483a-aa72-5d7fb6000bab None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:49.999652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 511/1007] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:10:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:50.338823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25512401-06af-489f-9ddf-36852b79d884 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:50.339347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25512401-06af-489f-9ddf-36852b79d884 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:50.339347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25512401-06af-489f-9ddf-36852b79d884 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:50.340753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 497/1008] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:50.346679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bd949-d214-49f2-9a72-54b207ef8399 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:50.346964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bd949-d214-49f2-9a72-54b207ef8399 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:50.347235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bd949-d214-49f2-9a72-54b207ef8399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:50.365658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5f4bd949-d214-49f2-9a72-54b207ef8399 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:50.366824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5f4bd949-d214-49f2-9a72-54b207ef8399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:50.439565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 512/1009] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:50 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:50.706783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25512401-06af-489f-9ddf-36852b79d884 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:50.738831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:50.810476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25512401-06af-489f-9ddf-36852b79d884 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:50.844243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78461b-dd9c-4be7-9175-46bc38f5d44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:50.845419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78461b-dd9c-4be7-9175-46bc38f5d44f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:50.845694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78461b-dd9c-4be7-9175-46bc38f5d44f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:50.845961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78461b-dd9c-4be7-9175-46bc38f5d44f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:50.846531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae78461b-dd9c-4be7-9175-46bc38f5d44f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:50.847913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae78461b-dd9c-4be7-9175-46bc38f5d44f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-ae78461b-dd9c-4be7-9175-46bc38f5d44f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:50.856467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:50.857479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 12:10:50.858341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:50.862429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 498/1010] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:50.863866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bd949-d214-49f2-9a72-54b207ef8399 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:50.962886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bd949-d214-49f2-9a72-54b207ef8399 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:50.973338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:50.998072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7f805-5e4a-4139-bdcc-8979c73c0156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:50.998674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7f805-5e4a-4139-bdcc-8979c73c0156 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:50.998946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7f805-5e4a-4139-bdcc-8979c73c0156 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:50.999211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7f805-5e4a-4139-bdcc-8979c73c0156 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:50.999864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7f805-5e4a-4139-bdcc-8979c73c0156 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:51.000427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7f805-5e4a-4139-bdcc-8979c73c0156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:51.002126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7f805-5e4a-4139-bdcc-8979c73c0156 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:51.011528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 513/1011] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:51.012613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78461b-dd9c-4be7-9175-46bc38f5d44f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:51.122418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:51.123930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53d476-0f78-4736-9056-826d41d7e139 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:51.124420 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53d476-0f78-4736-9056-826d41d7e139 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:51.124531 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53d476-0f78-4736-9056-826d41d7e139 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:51.130125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 514/1012] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:51.131158 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78461b-dd9c-4be7-9175-46bc38f5d44f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:51.137326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc105c6-6558-46cc-9afd-7116b2c098a5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:51.137326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc105c6-6558-46cc-9afd-7116b2c098a5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:51.137326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc105c6-6558-46cc-9afd-7116b2c098a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:51.154119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bdc105c6-6558-46cc-9afd-7116b2c098a5 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:51.156050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bdc105c6-6558-46cc-9afd-7116b2c098a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:51.171176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062ccd6-071c-4eb1-bdf7-45d270b345b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:51.171812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062ccd6-071c-4eb1-bdf7-45d270b345b0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:51.172096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062ccd6-071c-4eb1-bdf7-45d270b345b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:51.172368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062ccd6-071c-4eb1-bdf7-45d270b345b0 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:51.173051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062ccd6-071c-4eb1-bdf7-45d270b345b0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:51.173353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062ccd6-071c-4eb1-bdf7-45d270b345b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:51.174717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062ccd6-071c-4eb1-bdf7-45d270b345b0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:51.180184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 499/1013] 10.222.0.62 () {56 vars in 1112 bytes} [Mon Sep 5 12:10:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:51.185446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062ccd6-071c-4eb1-bdf7-45d270b345b0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:51.257127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 515/1014] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:51.293686 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062ccd6-071c-4eb1-bdf7-45d270b345b0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:51.329124 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59281c-31ac-4f07-9850-82ae8434cc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:51.329732 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59281c-31ac-4f07-9850-82ae8434cc57 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:51.330013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59281c-31ac-4f07-9850-82ae8434cc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:51.330296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59281c-31ac-4f07-9850-82ae8434cc57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:51.330955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59281c-31ac-4f07-9850-82ae8434cc57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:51.332390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59281c-31ac-4f07-9850-82ae8434cc57 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:51.429825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 500/1015] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:10:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:10:51.532689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc105c6-6558-46cc-9afd-7116b2c098a5 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:51.638411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc105c6-6558-46cc-9afd-7116b2c098a5 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:51.639988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:51.682549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885741ce-8abb-43e6-9130-88ddc0532baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:51.683160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885741ce-8abb-43e6-9130-88ddc0532baa None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:51.683438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885741ce-8abb-43e6-9130-88ddc0532baa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:51.683739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885741ce-8abb-43e6-9130-88ddc0532baa None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:51.684393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885741ce-8abb-43e6-9130-88ddc0532baa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:51.685741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885741ce-8abb-43e6-9130-88ddc0532baa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:51.765026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 516/1016] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:10:51 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:51.903300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406044c-996a-4be0-ad18-036213206961 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:51.903482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406044c-996a-4be0-ad18-036213206961 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:51.903799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406044c-996a-4be0-ad18-036213206961 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:51.905134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 501/1017] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:51.910912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e534-0174-4e22-946d-70ff4ed66de1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:51.911198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e534-0174-4e22-946d-70ff4ed66de1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:51.911474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e534-0174-4e22-946d-70ff4ed66de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:51.931567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5c22e534-0174-4e22-946d-70ff4ed66de1 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:51.932856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5c22e534-0174-4e22-946d-70ff4ed66de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:52.008255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 517/1018] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:52.274154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406044c-996a-4be0-ad18-036213206961 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:52.291293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:52.375657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406044c-996a-4be0-ad18-036213206961 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:52.410706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc28c03-522f-4fc2-9fb6-3fd3bc7b9834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:52.411328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc28c03-522f-4fc2-9fb6-3fd3bc7b9834 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:52.411697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc28c03-522f-4fc2-9fb6-3fd3bc7b9834 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:52.411969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc28c03-522f-4fc2-9fb6-3fd3bc7b9834 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:52.413060 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc28c03-522f-4fc2-9fb6-3fd3bc7b9834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:52.414591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc28c03-522f-4fc2-9fb6-3fd3bc7b9834 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:52.416395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 502/1019] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:52.421895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22e534-0174-4e22-946d-70ff4ed66de1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:52.508079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:52.523615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22e534-0174-4e22-946d-70ff4ed66de1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:52.556895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8248ca8-543b-423d-b625-90b9c3639037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:52.557428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8248ca8-543b-423d-b625-90b9c3639037 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:52.557702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8248ca8-543b-423d-b625-90b9c3639037 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:52.557971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8248ca8-543b-423d-b625-90b9c3639037 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:52.558619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8248ca8-543b-423d-b625-90b9c3639037 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:52.560174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8248ca8-543b-423d-b625-90b9c3639037 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:52.640148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 518/1020] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:10:52 2022] POST /identity/v3/projects => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:10:53.432058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f421fc-3389-48a5-bd09-1fe0152bc88c None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:53.432558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f421fc-3389-48a5-bd09-1fe0152bc88c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:53.432558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f421fc-3389-48a5-bd09-1fe0152bc88c None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:53.434030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 503/1021] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:53.438983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d824f3f-3ddb-46dc-8549-b4fc668275f4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:53.439363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d824f3f-3ddb-46dc-8549-b4fc668275f4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:53.439524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d824f3f-3ddb-46dc-8549-b4fc668275f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:53.458427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4d824f3f-3ddb-46dc-8549-b4fc668275f4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:53.459627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4d824f3f-3ddb-46dc-8549-b4fc668275f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:53.527778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 519/1022] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:53.695507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf5f19-9a8c-49a1-80db-8828b164b338 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:53.695716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf5f19-9a8c-49a1-80db-8828b164b338 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:53.695994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf5f19-9a8c-49a1-80db-8828b164b338 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:53.697490 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 504/1023] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:53.702795 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116b7b-6aa0-450f-9aee-5632734f0e49 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:53.703075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116b7b-6aa0-450f-9aee-5632734f0e49 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:53.703351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116b7b-6aa0-450f-9aee-5632734f0e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:53.722747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d4116b7b-6aa0-450f-9aee-5632734f0e49 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:53.723953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d4116b7b-6aa0-450f-9aee-5632734f0e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:53.788288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:53.799626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 520/1024] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:53.809391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf5f19-9a8c-49a1-80db-8828b164b338 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:53.882753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:53.902306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf5f19-9a8c-49a1-80db-8828b164b338 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:53.936095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1b822-3ac5-4755-9c4b-fca008cfc9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:53.936711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1b822-3ac5-4755-9c4b-fca008cfc9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:53.936988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1b822-3ac5-4755-9c4b-fca008cfc9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:53.937256 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1b822-3ac5-4755-9c4b-fca008cfc9b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:53.948109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1b822-3ac5-4755-9c4b-fca008cfc9b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-dda1b822-3ac5-4755-9c4b-fca008cfc9b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:53.952779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:53.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:10:53.955193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:53.959242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 505/1025] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Sep 5 12:10:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:53.960487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116b7b-6aa0-450f-9aee-5632734f0e49 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.068222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116b7b-6aa0-450f-9aee-5632734f0e49 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.090426 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1b822-3ac5-4755-9c4b-fca008cfc9b2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.106858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfaaae-cdc2-4e13-a380-04bedb4e625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.107458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfaaae-cdc2-4e13-a380-04bedb4e625b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.107770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfaaae-cdc2-4e13-a380-04bedb4e625b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.108273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfaaae-cdc2-4e13-a380-04bedb4e625b None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.108690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfaaae-cdc2-4e13-a380-04bedb4e625b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:54.108988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfaaae-cdc2-4e13-a380-04bedb4e625b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.110367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfaaae-cdc2-4e13-a380-04bedb4e625b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.118971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 521/1026] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Sep 5 12:10:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:54.124792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfaaae-cdc2-4e13-a380-04bedb4e625b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.187498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.202846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1b822-3ac5-4755-9c4b-fca008cfc9b2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.231497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfaaae-cdc2-4e13-a380-04bedb4e625b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.235083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06779d-56ae-4812-87f7-2ac67137cb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.236315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06779d-56ae-4812-87f7-2ac67137cb6a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.236595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06779d-56ae-4812-87f7-2ac67137cb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.236864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06779d-56ae-4812-87f7-2ac67137cb6a None admin] PATH_INFO: `/v3/services/image` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.237441 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06779d-56ae-4812-87f7-2ac67137cb6a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.238826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06779d-56ae-4812-87f7-2ac67137cb6a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-8e06779d-56ae-4812-87f7-2ac67137cb6a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:54.247866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:54.248890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:54.249750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 506/1027] 10.222.0.62 () {56 vars in 1105 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:54.254927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06779d-56ae-4812-87f7-2ac67137cb6a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.263767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.273346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd0ab6-cc1d-4aa7-99a3-9eddb7dec63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.278311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd0ab6-cc1d-4aa7-99a3-9eddb7dec63a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.278311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd0ab6-cc1d-4aa7-99a3-9eddb7dec63a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.278311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd0ab6-cc1d-4aa7-99a3-9eddb7dec63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.286508 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd0ab6-cc1d-4aa7-99a3-9eddb7dec63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.288085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd0ab6-cc1d-4aa7-99a3-9eddb7dec63a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e1fd0ab6-cc1d-4aa7-99a3-9eddb7dec63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:54.304385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:54.305515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:54.306644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:54.306644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:54.306644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:54.308018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 522/1028] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:54.313818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd0ab6-cc1d-4aa7-99a3-9eddb7dec63a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.364082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06779d-56ae-4812-87f7-2ac67137cb6a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.398874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ec82-54cb-468f-b23b-a2b557749ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.399213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ec82-54cb-468f-b23b-a2b557749ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.399492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ec82-54cb-468f-b23b-a2b557749ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.399789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ec82-54cb-468f-b23b-a2b557749ee5 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.400412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89ec82-54cb-468f-b23b-a2b557749ee5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:54.400845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89ec82-54cb-468f-b23b-a2b557749ee5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.402070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89ec82-54cb-468f-b23b-a2b557749ee5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.408147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 507/1029] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:54.414385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ec82-54cb-468f-b23b-a2b557749ee5 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.423291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd0ab6-cc1d-4aa7-99a3-9eddb7dec63a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.435471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.459995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d75fcb-0ba9-4a66-ae40-14f735342ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.460719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d75fcb-0ba9-4a66-ae40-14f735342ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.461156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d75fcb-0ba9-4a66-ae40-14f735342ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.461656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d75fcb-0ba9-4a66-ae40-14f735342ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.462828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d75fcb-0ba9-4a66-ae40-14f735342ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:54.463230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d75fcb-0ba9-4a66-ae40-14f735342ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.464632 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d75fcb-0ba9-4a66-ae40-14f735342ee2 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.477989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 523/1030] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:54.483409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d75fcb-0ba9-4a66-ae40-14f735342ee2 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.490264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.521915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ec82-54cb-468f-b23b-a2b557749ee5 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.558645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84516e55-a76b-40ef-adf7-cabfbb7381db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.559247 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84516e55-a76b-40ef-adf7-cabfbb7381db None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.559526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84516e55-a76b-40ef-adf7-cabfbb7381db None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.559826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84516e55-a76b-40ef-adf7-cabfbb7381db None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.560452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84516e55-a76b-40ef-adf7-cabfbb7381db None admin] RBAC: Adding query filter params (type=image) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:54.560751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84516e55-a76b-40ef-adf7-cabfbb7381db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.562115 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84516e55-a76b-40ef-adf7-cabfbb7381db None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.572938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 508/1031] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:54.574195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84516e55-a76b-40ef-adf7-cabfbb7381db None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.590414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d75fcb-0ba9-4a66-ae40-14f735342ee2 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.626459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c719f7-d7c4-434b-880d-3e90a39acfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.627135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c719f7-d7c4-434b-880d-3e90a39acfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.627415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c719f7-d7c4-434b-880d-3e90a39acfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.627716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c719f7-d7c4-434b-880d-3e90a39acfb5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.634599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c719f7-d7c4-434b-880d-3e90a39acfb5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.636070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c719f7-d7c4-434b-880d-3e90a39acfb5 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-c2c719f7-d7c4-434b-880d-3e90a39acfb5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:54.647970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:54.649127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:54.650240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:54.650240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:54.650240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:54.650240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:54.650240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:54.650240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:54.650240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:54.650240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:54.650240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:54.650240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:54.651696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 524/1032] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:54.652874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2686a4-c4f8-4b79-9b83-30bfcf7e6c91 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.653341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2686a4-c4f8-4b79-9b83-30bfcf7e6c91 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.653466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2686a4-c4f8-4b79-9b83-30bfcf7e6c91 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.654645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 525/1033] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:54.655325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2686a4-c4f8-4b79-9b83-30bfcf7e6c91 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.677503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84516e55-a76b-40ef-adf7-cabfbb7381db None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.686083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.711018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de5765-1212-4a11-a91c-3225cb1899ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.711577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de5765-1212-4a11-a91c-3225cb1899ea None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.711862 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de5765-1212-4a11-a91c-3225cb1899ea None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.712133 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de5765-1212-4a11-a91c-3225cb1899ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.712824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de5765-1212-4a11-a91c-3225cb1899ea None admin] RBAC: Adding query filter params (interface=public, service_id=7efd4d8d627f4014a1f2b9d8aeaee6cb, region_id=RegionOne) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:54.713128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de5765-1212-4a11-a91c-3225cb1899ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.714492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de5765-1212-4a11-a91c-3225cb1899ea None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.715965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.720373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 509/1034] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/endpoints?service_id=7efd4d8d627f4014a1f2b9d8aeaee6cb&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:54.721518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba341e9-07a4-46ef-83d3-902c6c05c865 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.721802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba341e9-07a4-46ef-83d3-902c6c05c865 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.722092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba341e9-07a4-46ef-83d3-902c6c05c865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.739961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1ba341e9-07a4-46ef-83d3-902c6c05c865 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:54.741230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1ba341e9-07a4-46ef-83d3-902c6c05c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:54.758066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2686a4-c4f8-4b79-9b83-30bfcf7e6c91 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.790932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d699f4-4be3-4e23-b1e4-5b17520600b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.791561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d699f4-4be3-4e23-b1e4-5b17520600b3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.791835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d699f4-4be3-4e23-b1e4-5b17520600b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.792100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d699f4-4be3-4e23-b1e4-5b17520600b3 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.792845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d699f4-4be3-4e23-b1e4-5b17520600b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:54.793143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d699f4-4be3-4e23-b1e4-5b17520600b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.794540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d699f4-4be3-4e23-b1e4-5b17520600b3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.807111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 510/1035] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:54.808032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba341e9-07a4-46ef-83d3-902c6c05c865 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.815859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 526/1036] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:54.827585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d699f4-4be3-4e23-b1e4-5b17520600b3 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:54.914598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba341e9-07a4-46ef-83d3-902c6c05c865 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.921687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.933815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d699f4-4be3-4e23-b1e4-5b17520600b3 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:54.944590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:54.952376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216d606-e88f-4268-9284-ac0e02225fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.953064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216d606-e88f-4268-9284-ac0e02225fec None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.953344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216d606-e88f-4268-9284-ac0e02225fec None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.953619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216d606-e88f-4268-9284-ac0e02225fec None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:54.954816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216d606-e88f-4268-9284-ac0e02225fec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:54.956228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216d606-e88f-4268-9284-ac0e02225fec None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:54.962944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 511/1037] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:10:54.976834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a1380-d42c-4fe4-9fa6-a5b19518f486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:54.977459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a1380-d42c-4fe4-9fa6-a5b19518f486 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:54.977733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a1380-d42c-4fe4-9fa6-a5b19518f486 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:54.978000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a1380-d42c-4fe4-9fa6-a5b19518f486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:55.059122 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a1380-d42c-4fe4-9fa6-a5b19518f486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:55.060631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a1380-d42c-4fe4-9fa6-a5b19518f486 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:55.086866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216d606-e88f-4268-9284-ac0e02225fec None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:55.156635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:55.169277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 527/1038] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:54 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:10:55.175434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a1380-d42c-4fe4-9fa6-a5b19518f486 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:55.209573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216d606-e88f-4268-9284-ac0e02225fec None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:55.244444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4e6d4-8e54-4efd-b777-ec568873dd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:55.245002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4e6d4-8e54-4efd-b777-ec568873dd10 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:55.245291 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4e6d4-8e54-4efd-b777-ec568873dd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:55.245564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4e6d4-8e54-4efd-b777-ec568873dd10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:55.246668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4e6d4-8e54-4efd-b777-ec568873dd10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:55.248141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4e6d4-8e54-4efd-b777-ec568873dd10 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:55.249878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 512/1039] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:55.255911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4e6d4-8e54-4efd-b777-ec568873dd10 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:55.280533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a1380-d42c-4fe4-9fa6-a5b19518f486 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:55.317242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d467a31-81ed-41eb-9aa2-969c9fd5f864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:55.317863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d467a31-81ed-41eb-9aa2-969c9fd5f864 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:55.318138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d467a31-81ed-41eb-9aa2-969c9fd5f864 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:55.318407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d467a31-81ed-41eb-9aa2-969c9fd5f864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:55.322979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:55.328862 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d467a31-81ed-41eb-9aa2-969c9fd5f864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:55.330295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d467a31-81ed-41eb-9aa2-969c9fd5f864 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-5d467a31-81ed-41eb-9aa2-969c9fd5f864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:55.346023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:55.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:55.348156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:55.348156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:55.348156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:55.348156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 528/1040] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:55.352613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d467a31-81ed-41eb-9aa2-969c9fd5f864 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:55.369663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4e6d4-8e54-4efd-b777-ec568873dd10 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:55.394692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:55.407784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd70764-96ee-48db-92fc-00260ab4c86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:55.408633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd70764-96ee-48db-92fc-00260ab4c86e None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:55.409056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd70764-96ee-48db-92fc-00260ab4c86e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:55.409316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd70764-96ee-48db-92fc-00260ab4c86e None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:55.409960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd70764-96ee-48db-92fc-00260ab4c86e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:55.411393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd70764-96ee-48db-92fc-00260ab4c86e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:55.459620 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d467a31-81ed-41eb-9aa2-969c9fd5f864 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:55.493775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfcb06-7e82-4ffb-962a-de7fa68169df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:55.494398 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfcb06-7e82-4ffb-962a-de7fa68169df None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:55.494676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfcb06-7e82-4ffb-962a-de7fa68169df None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:55.494895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfcb06-7e82-4ffb-962a-de7fa68169df None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:55.495527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfcb06-7e82-4ffb-962a-de7fa68169df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:55.495849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfcb06-7e82-4ffb-962a-de7fa68169df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:55.497203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfcb06-7e82-4ffb-962a-de7fa68169df None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:55.508943 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 529/1041] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:55.523632 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 513/1042] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:10:55 2022] POST /identity/v3/users => generated 285 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:10:55.524216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfcb06-7e82-4ffb-962a-de7fa68169df None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:55.621523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:55.625196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfcb06-7e82-4ffb-962a-de7fa68169df None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:55.663086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa33d7-7a10-4dc5-b9fb-5171596ac852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:55.663753 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa33d7-7a10-4dc5-b9fb-5171596ac852 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:55.664028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa33d7-7a10-4dc5-b9fb-5171596ac852 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:55.665317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa33d7-7a10-4dc5-b9fb-5171596ac852 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:55.671025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa33d7-7a10-4dc5-b9fb-5171596ac852 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:55.672502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa33d7-7a10-4dc5-b9fb-5171596ac852 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-79aa33d7-7a10-4dc5-b9fb-5171596ac852 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:55.684200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:55.685255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:55.686356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:55.686356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:55.686356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:55.686356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:55.686356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:55.686356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:55.686356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:55.686356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:55.690164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 530/1043] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:55.691514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd70764-96ee-48db-92fc-00260ab4c86e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:55.795145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd70764-96ee-48db-92fc-00260ab4c86e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:55.811647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:55.829859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6173e8d-9b25-4088-be60-e08ce6fab480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:55.830464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6173e8d-9b25-4088-be60-e08ce6fab480 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:55.830736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6173e8d-9b25-4088-be60-e08ce6fab480 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:55.831001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6173e8d-9b25-4088-be60-e08ce6fab480 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:55.831708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6173e8d-9b25-4088-be60-e08ce6fab480 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:55.832006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6173e8d-9b25-4088-be60-e08ce6fab480 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:55.833431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6173e8d-9b25-4088-be60-e08ce6fab480 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:55.847341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 514/1044] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:55.853088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa33d7-7a10-4dc5-b9fb-5171596ac852 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:55.956202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa33d7-7a10-4dc5-b9fb-5171596ac852 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:55.990560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2da77-7c74-4fde-b228-76d74efa2068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:55.991203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2da77-7c74-4fde-b228-76d74efa2068 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:55.991467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2da77-7c74-4fde-b228-76d74efa2068 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:55.991775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2da77-7c74-4fde-b228-76d74efa2068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:55.992493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2da77-7c74-4fde-b228-76d74efa2068 None admin] RBAC: Adding query filter params (role.id=8b2491af21004040878bb0b8acb0194d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=7956a7748ec94a618eb466f5536220b0) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:55.992797 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2da77-7c74-4fde-b228-76d74efa2068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:55.994201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2da77-7c74-4fde-b228-76d74efa2068 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:56.007808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 531/1045] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:55 2022] GET /identity/v3/role_assignments?user.id=7956a7748ec94a618eb466f5536220b0&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=8b2491af21004040878bb0b8acb0194d => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:56.881774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c77ab-042f-44d6-a2ca-a08cf9f119fc None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:56.882295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c77ab-042f-44d6-a2ca-a08cf9f119fc None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:56.882295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c77ab-042f-44d6-a2ca-a08cf9f119fc None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:56.883714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 515/1046] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:56.888755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91c6ab-e385-4472-b149-49faf1ff18aa None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:56.889193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91c6ab-e385-4472-b149-49faf1ff18aa None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:56.889286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91c6ab-e385-4472-b149-49faf1ff18aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:56.907943 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-cc91c6ab-e385-4472-b149-49faf1ff18aa None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:56.909111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cc91c6ab-e385-4472-b149-49faf1ff18aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:56.932005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:56.981298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 532/1047] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:10:57.246008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c77ab-042f-44d6-a2ca-a08cf9f119fc None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:57.349697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c77ab-042f-44d6-a2ca-a08cf9f119fc None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:57.383055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6487ea8-c53e-42e7-81c2-ae6a485198d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:57.383706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6487ea8-c53e-42e7-81c2-ae6a485198d5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:57.384033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6487ea8-c53e-42e7-81c2-ae6a485198d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:57.384307 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6487ea8-c53e-42e7-81c2-ae6a485198d5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:57.384875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6487ea8-c53e-42e7-81c2-ae6a485198d5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:57.386228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6487ea8-c53e-42e7-81c2-ae6a485198d5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-d6487ea8-c53e-42e7-81c2-ae6a485198d5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:57.395465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:57.396569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 12:10:57.398010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:57.399171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 516/1048] 10.222.0.62 () {56 vars in 1105 bytes} [Mon Sep 5 12:10:57 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:57.402417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91c6ab-e385-4472-b149-49faf1ff18aa None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:57.503088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91c6ab-e385-4472-b149-49faf1ff18aa None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:57.515953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9df37-e71e-49a1-bfa4-50667040331f None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:57.516239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9df37-e71e-49a1-bfa4-50667040331f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:57.516520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9df37-e71e-49a1-bfa4-50667040331f None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:57.518535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 517/1049] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:57.523177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab3398-5f70-42cb-ba52-457431ab407f None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:57.523467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab3398-5f70-42cb-ba52-457431ab407f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:57.523772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab3398-5f70-42cb-ba52-457431ab407f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:57.539641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a5708-2c37-47f6-8593-aa824b5ba37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:57.540259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a5708-2c37-47f6-8593-aa824b5ba37e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:57.540536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a5708-2c37-47f6-8593-aa824b5ba37e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:57.540807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a5708-2c37-47f6-8593-aa824b5ba37e None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:57.541440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a5708-2c37-47f6-8593-aa824b5ba37e None admin] RBAC: Adding query filter params (name=image) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:57.541747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a5708-2c37-47f6-8593-aa824b5ba37e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:57.542550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-0aab3398-5f70-42cb-ba52-457431ab407f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:57.543114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a5708-2c37-47f6-8593-aa824b5ba37e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:57.543777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0aab3398-5f70-42cb-ba52-457431ab407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:57.549024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 533/1050] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:57.554443 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a5708-2c37-47f6-8593-aa824b5ba37e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:57.573062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:57.581429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:57.615874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 518/1051] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:57.664472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a5708-2c37-47f6-8593-aa824b5ba37e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:57.699361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012bc859-ed13-4011-8748-1652ace6c674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:57.699992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012bc859-ed13-4011-8748-1652ace6c674 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:57.700268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012bc859-ed13-4011-8748-1652ace6c674 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:57.700537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012bc859-ed13-4011-8748-1652ace6c674 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:57.701168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012bc859-ed13-4011-8748-1652ace6c674 None admin] RBAC: Adding query filter params (type=image) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:57.701467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012bc859-ed13-4011-8748-1652ace6c674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:57.702810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012bc859-ed13-4011-8748-1652ace6c674 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:57.713129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 534/1052] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:57.713988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab3398-5f70-42cb-ba52-457431ab407f None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:57.818684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab3398-5f70-42cb-ba52-457431ab407f None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:57.852680 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5e449-efad-4032-a9ad-064615061f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:57.853290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5e449-efad-4032-a9ad-064615061f23 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:57.853567 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5e449-efad-4032-a9ad-064615061f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:57.853840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5e449-efad-4032-a9ad-064615061f23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:57.854499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5e449-efad-4032-a9ad-064615061f23 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:57.855891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5e449-efad-4032-a9ad-064615061f23 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:57.885477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012bc859-ed13-4011-8748-1652ace6c674 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:57.942372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 519/1053] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:10:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:10:57.943482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf1e74-622b-4e97-9649-e2d3c3bd0f3b None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:57.943795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf1e74-622b-4e97-9649-e2d3c3bd0f3b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:57.944115 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf1e74-622b-4e97-9649-e2d3c3bd0f3b None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:57.945262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 520/1054] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:57.950741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4df975-16b1-4eb7-90da-06433867c5fb None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:57.951015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4df975-16b1-4eb7-90da-06433867c5fb None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:57.951279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4df975-16b1-4eb7-90da-06433867c5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:57.971320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:57.971906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-3d4df975-16b1-4eb7-90da-06433867c5fb None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:57.973046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3d4df975-16b1-4eb7-90da-06433867c5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:57.993870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012bc859-ed13-4011-8748-1652ace6c674 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:57.996202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:58.039787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be58ca4-20d4-441a-b376-f4678e354483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:58.040425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be58ca4-20d4-441a-b376-f4678e354483 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:58.040702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be58ca4-20d4-441a-b376-f4678e354483 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:58.040979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be58ca4-20d4-441a-b376-f4678e354483 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:58.049868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 521/1055] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:10:58.055563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be58ca4-20d4-441a-b376-f4678e354483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-8be58ca4-20d4-441a-b376-f4678e354483 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:58.060540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:58.061534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:58.068800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 535/1056] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:10:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:58.069532 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4df975-16b1-4eb7-90da-06433867c5fb None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:58.171427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4df975-16b1-4eb7-90da-06433867c5fb None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:58.207054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447707a-eb99-4da2-9f08-1d03d505b3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:58.207696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447707a-eb99-4da2-9f08-1d03d505b3ab None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:58.207981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447707a-eb99-4da2-9f08-1d03d505b3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:58.208252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447707a-eb99-4da2-9f08-1d03d505b3ab None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:58.208911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447707a-eb99-4da2-9f08-1d03d505b3ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:58.209212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447707a-eb99-4da2-9f08-1d03d505b3ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:58.210579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447707a-eb99-4da2-9f08-1d03d505b3ab None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:58.219412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 522/1057] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:10:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:58.225336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be58ca4-20d4-441a-b376-f4678e354483 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:58.319090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447707a-eb99-4da2-9f08-1d03d505b3ab None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:58.329629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be58ca4-20d4-441a-b376-f4678e354483 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:58.339472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:58.365966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d261f0-0fe1-4d61-a3c4-6cdb85b20353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:58.367991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d261f0-0fe1-4d61-a3c4-6cdb85b20353 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:58.367991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d261f0-0fe1-4d61-a3c4-6cdb85b20353 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:58.367991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d261f0-0fe1-4d61-a3c4-6cdb85b20353 None admin] PATH_INFO: `/v3/users/6c336f83df0145f2ba48dddd05ebd280` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:58.374739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:58.380914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d261f0-0fe1-4d61-a3c4-6cdb85b20353 None admin] RBAC: Authorizing `identity:get_user(user_id=6c336f83df0145f2ba48dddd05ebd280)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:58.382356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d261f0-0fe1-4d61-a3c4-6cdb85b20353 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:58.384120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 536/1058] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:10:58 2022] GET /identity/v3/users/6c336f83df0145f2ba48dddd05ebd280 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:58.389790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d261f0-0fe1-4d61-a3c4-6cdb85b20353 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:58.429508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447707a-eb99-4da2-9f08-1d03d505b3ab None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:58.489563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d261f0-0fe1-4d61-a3c4-6cdb85b20353 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:58.524011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943affd-0723-4864-bf99-aa4495a3d05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:58.524621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943affd-0723-4864-bf99-aa4495a3d05b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:58.524900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943affd-0723-4864-bf99-aa4495a3d05b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:58.525173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943affd-0723-4864-bf99-aa4495a3d05b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:58.528608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c16f9-f775-4278-873f-589d2080457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:58.529235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c16f9-f775-4278-873f-589d2080457e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:58.529512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c16f9-f775-4278-873f-589d2080457e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:58.529786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c16f9-f775-4278-873f-589d2080457e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:58.540424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c16f9-f775-4278-873f-589d2080457e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-199c16f9-f775-4278-873f-589d2080457e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:10:58.545120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:58.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:58.546906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 523/1059] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Sep 5 12:10:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:58.552578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c16f9-f775-4278-873f-589d2080457e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:58.602204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:58.610291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943affd-0723-4864-bf99-aa4495a3d05b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:58.617457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943affd-0723-4864-bf99-aa4495a3d05b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:58.633024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:58.655045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c16f9-f775-4278-873f-589d2080457e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:58.691903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f61b2-1c98-4a98-83c3-58dd78f0b4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:58.692507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f61b2-1c98-4a98-83c3-58dd78f0b4cd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:58.692789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f61b2-1c98-4a98-83c3-58dd78f0b4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:58.693063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f61b2-1c98-4a98-83c3-58dd78f0b4cd None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:58.693846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f61b2-1c98-4a98-83c3-58dd78f0b4cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:58.694148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f61b2-1c98-4a98-83c3-58dd78f0b4cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:58.695702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f61b2-1c98-4a98-83c3-58dd78f0b4cd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:58.704198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 524/1060] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Sep 5 12:10:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:58.710069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f61b2-1c98-4a98-83c3-58dd78f0b4cd None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:58.716899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 537/1061] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:10:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:10:58.728752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943affd-0723-4864-bf99-aa4495a3d05b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:58.823972 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f61b2-1c98-4a98-83c3-58dd78f0b4cd None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:58.828538 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:58.834828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943affd-0723-4864-bf99-aa4495a3d05b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:58.862768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48501fc8-778f-4957-bb75-2331d00e46fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:58.863323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48501fc8-778f-4957-bb75-2331d00e46fb None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:58.863721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48501fc8-778f-4957-bb75-2331d00e46fb None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:58.863965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48501fc8-778f-4957-bb75-2331d00e46fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:58.869910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eafe22-8003-435c-95d8-c0aa7435a5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:58.870536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eafe22-8003-435c-95d8-c0aa7435a5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:58.870810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eafe22-8003-435c-95d8-c0aa7435a5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:58.871085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eafe22-8003-435c-95d8-c0aa7435a5e1 None admin] PATH_INFO: `/v3/projects/8fa602f1cad549258cea821337eb25b1` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:58.872250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eafe22-8003-435c-95d8-c0aa7435a5e1 None admin] RBAC: Authorizing `identity:get_project(project_id=8fa602f1cad549258cea821337eb25b1)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:58.872675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:58.873745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eafe22-8003-435c-95d8-c0aa7435a5e1 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:58.874883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48501fc8-778f-4957-bb75-2331d00e46fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:58.876364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48501fc8-778f-4957-bb75-2331d00e46fb None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:58.880623 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 538/1062] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:10:58 2022] GET /identity/v3/projects/8fa602f1cad549258cea821337eb25b1 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:58.882361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eafe22-8003-435c-95d8-c0aa7435a5e1 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-48501fc8-778f-4957-bb75-2331d00e46fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:58.890477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:58.891481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:58.893977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:58.893977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:58.893977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:10:58.893977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 525/1063] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:58.897751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48501fc8-778f-4957-bb75-2331d00e46fb None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:58.990492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eafe22-8003-435c-95d8-c0aa7435a5e1 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:59.006772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48501fc8-778f-4957-bb75-2331d00e46fb None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:59.026145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf7ff8-5e95-46d3-a375-8a9170343461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:59.026750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf7ff8-5e95-46d3-a375-8a9170343461 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:59.027025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf7ff8-5e95-46d3-a375-8a9170343461 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:59.027294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf7ff8-5e95-46d3-a375-8a9170343461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:59.028167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf7ff8-5e95-46d3-a375-8a9170343461 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=8fa602f1cad549258cea821337eb25b1, user.id=6c336f83df0145f2ba48dddd05ebd280) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:59.028470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf7ff8-5e95-46d3-a375-8a9170343461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:59.029873 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf7ff8-5e95-46d3-a375-8a9170343461 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:59.042841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981455c-f896-4a3c-b060-6de2c8706a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:59.043484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981455c-f896-4a3c-b060-6de2c8706a98 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:59.043797 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981455c-f896-4a3c-b060-6de2c8706a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:59.044072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981455c-f896-4a3c-b060-6de2c8706a98 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:59.044751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981455c-f896-4a3c-b060-6de2c8706a98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:59.045057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981455c-f896-4a3c-b060-6de2c8706a98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:59.047613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981455c-f896-4a3c-b060-6de2c8706a98 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:59.049181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 539/1064] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:10:58 2022] GET /identity/v3/role_assignments?user.id=6c336f83df0145f2ba48dddd05ebd280&scope.project.id=8fa602f1cad549258cea821337eb25b1&role.id=705907b9bdec47cebf88b868a6a56356 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:59.068964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 526/1065] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:59.070430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf7ff8-5e95-46d3-a375-8a9170343461 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:59.108008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:59.188150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf7ff8-5e95-46d3-a375-8a9170343461 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:59.222865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9beb45f-cbc8-4438-b904-a7e0ced92207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:59.223497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beb45f-cbc8-4438-b904-a7e0ced92207 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:59.223813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beb45f-cbc8-4438-b904-a7e0ced92207 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:59.224084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beb45f-cbc8-4438-b904-a7e0ced92207 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:59.231453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9beb45f-cbc8-4438-b904-a7e0ced92207 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:59.232898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9beb45f-cbc8-4438-b904-a7e0ced92207 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-d9beb45f-cbc8-4438-b904-a7e0ced92207 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:10:59.243877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:59.244809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:59.245748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 540/1066] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Sep 5 12:10:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:59.250853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981455c-f896-4a3c-b060-6de2c8706a98 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:59.306411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:59.354611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981455c-f896-4a3c-b060-6de2c8706a98 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:59.390152 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d5931-acd8-4ac9-851f-d98e4ad83327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:59.390766 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d5931-acd8-4ac9-851f-d98e4ad83327 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:59.391047 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d5931-acd8-4ac9-851f-d98e4ad83327 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:59.391360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d5931-acd8-4ac9-851f-d98e4ad83327 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:59.392678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d5931-acd8-4ac9-851f-d98e4ad83327 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:59.392983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d5931-acd8-4ac9-851f-d98e4ad83327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:59.394402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d5931-acd8-4ac9-851f-d98e4ad83327 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:59.416722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 527/1067] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Sep 5 12:10:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:10:59.418162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9beb45f-cbc8-4438-b904-a7e0ced92207 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:59.542660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:59.555343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9beb45f-cbc8-4438-b904-a7e0ced92207 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:59.589480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2841af4-61f9-4a42-990c-c6beb178c20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:59.590105 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841af4-61f9-4a42-990c-c6beb178c20a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:59.590380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841af4-61f9-4a42-990c-c6beb178c20a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:59.590647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841af4-61f9-4a42-990c-c6beb178c20a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:59.601405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2841af4-61f9-4a42-990c-c6beb178c20a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:59.602828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2841af4-61f9-4a42-990c-c6beb178c20a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-a2841af4-61f9-4a42-990c-c6beb178c20a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:10:59.617145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:59.618087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:10:59.618994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:10:59.618994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:10:59.618994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:59.618994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 541/1068] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:10:59.625307 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d5931-acd8-4ac9-851f-d98e4ad83327 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:59.724425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d5931-acd8-4ac9-851f-d98e4ad83327 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:59.734814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:59.759089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319ecda-8dc4-4293-bdb9-2294977db87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:59.759733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319ecda-8dc4-4293-bdb9-2294977db87e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:59.760017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319ecda-8dc4-4293-bdb9-2294977db87e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:59.760368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319ecda-8dc4-4293-bdb9-2294977db87e None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:59.761031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319ecda-8dc4-4293-bdb9-2294977db87e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:10:59.761371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319ecda-8dc4-4293-bdb9-2294977db87e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:59.762766 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319ecda-8dc4-4293-bdb9-2294977db87e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:59.779520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 528/1069] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:10:59.781220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2841af4-61f9-4a42-990c-c6beb178c20a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:10:59.883970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2841af4-61f9-4a42-990c-c6beb178c20a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:10:59.917912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f788bc4-8aeb-4335-8e76-0faf41fe375b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:10:59.918216 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd4d00-471f-491e-b46f-13b7ff2c7060 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:59.918511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd4d00-471f-491e-b46f-13b7ff2c7060 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:59.918627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f788bc4-8aeb-4335-8e76-0faf41fe375b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:59.918788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd4d00-471f-491e-b46f-13b7ff2c7060 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:59.918888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f788bc4-8aeb-4335-8e76-0faf41fe375b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:59.919102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f788bc4-8aeb-4335-8e76-0faf41fe375b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:59.920489 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 529/1070] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:10:59.925872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0013a64-fe46-40a6-8461-98f74a225e4a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:10:59.926103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0013a64-fe46-40a6-8461-98f74a225e4a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:10:59.926381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0013a64-fe46-40a6-8461-98f74a225e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:10:59.930148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f788bc4-8aeb-4335-8e76-0faf41fe375b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:10:59.931595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f788bc4-8aeb-4335-8e76-0faf41fe375b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:10:59.934535 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:10:59.944853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-f0013a64-fe46-40a6-8461-98f74a225e4a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-2f788bc4-8aeb-4335-8e76-0faf41fe375b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:10:59.947341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:10:59.949285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:10:59.950240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:10:59.950240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:10:59.950240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:10:59.950240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:10:59.950240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:10:59.950240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:10:59.950240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:10:59.950240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:10:59.950673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f0013a64-fe46-40a6-8461-98f74a225e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:10:59.950746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 542/1071] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:10:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:10:59.953046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f788bc4-8aeb-4335-8e76-0faf41fe375b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:00.022226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 530/1072] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:10:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:11:00.062922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f788bc4-8aeb-4335-8e76-0faf41fe375b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:00.102194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81feed-72b3-43a5-9a2b-3483ae5f12b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:00.102812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81feed-72b3-43a5-9a2b-3483ae5f12b0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:00.103090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81feed-72b3-43a5-9a2b-3483ae5f12b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:00.103362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81feed-72b3-43a5-9a2b-3483ae5f12b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:00.104096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81feed-72b3-43a5-9a2b-3483ae5f12b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:00.104399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81feed-72b3-43a5-9a2b-3483ae5f12b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:00.105824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81feed-72b3-43a5-9a2b-3483ae5f12b0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:00.118599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 543/1073] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:00.124592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0013a64-fe46-40a6-8461-98f74a225e4a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:00.222050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:00.225541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0013a64-fe46-40a6-8461-98f74a225e4a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:00.258626 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9027e58-574f-4aa1-bdbb-e8d9c551e451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:00.259278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9027e58-574f-4aa1-bdbb-e8d9c551e451 None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:00.259740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9027e58-574f-4aa1-bdbb-e8d9c551e451 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:00.260016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9027e58-574f-4aa1-bdbb-e8d9c551e451 None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/7956a7748ec94a618eb466f5536220b0/roles/8b2491af21004040878bb0b8acb0194d` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:00.262214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9027e58-574f-4aa1-bdbb-e8d9c551e451 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=7956a7748ec94a618eb466f5536220b0, role_id=8b2491af21004040878bb0b8acb0194d)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:00.263675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9027e58-574f-4aa1-bdbb-e8d9c551e451 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:00.295292 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81feed-72b3-43a5-9a2b-3483ae5f12b0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:00.357477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:00.397870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81feed-72b3-43a5-9a2b-3483ae5f12b0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:00.405834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 531/1074] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:11:00 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/7956a7748ec94a618eb466f5536220b0/roles/8b2491af21004040878bb0b8acb0194d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:11:00.436814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd472011-d1db-4f89-b369-0db555b27010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:00.437539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd472011-d1db-4f89-b369-0db555b27010 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:00.437885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd472011-d1db-4f89-b369-0db555b27010 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:00.438213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd472011-d1db-4f89-b369-0db555b27010 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:00.560548 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd472011-d1db-4f89-b369-0db555b27010 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:00.562054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd472011-d1db-4f89-b369-0db555b27010 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:00.620104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:00.662780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 544/1075] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:11:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:11:00.668550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9027e58-574f-4aa1-bdbb-e8d9c551e451 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:00.768574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9027e58-574f-4aa1-bdbb-e8d9c551e451 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:00.802791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cd380-4a4c-4aa5-b241-d2032c309512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:00.803412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cd380-4a4c-4aa5-b241-d2032c309512 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:00.803733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cd380-4a4c-4aa5-b241-d2032c309512 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:00.804004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cd380-4a4c-4aa5-b241-d2032c309512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:00.814107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cd380-4a4c-4aa5-b241-d2032c309512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:00.815486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cd380-4a4c-4aa5-b241-d2032c309512 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-5f0cd380-4a4c-4aa5-b241-d2032c309512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:11:00.830042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:00.831005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:11:00.832314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:11:00.832314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:11:00.832314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:00.836126 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 532/1076] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:00.837477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd472011-d1db-4f89-b369-0db555b27010 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:00.939042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd472011-d1db-4f89-b369-0db555b27010 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:00.966669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0851d-f642-43f6-97a2-003bac44c999 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:00.966957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0851d-f642-43f6-97a2-003bac44c999 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:00.967315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0851d-f642-43f6-97a2-003bac44c999 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:00.969012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 533/1077] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:11:00.973981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be62a8-5236-4405-928e-bb6a522de344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:00.974144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac76081-007e-4d05-a2b7-d9e7c37d5148 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:00.974376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac76081-007e-4d05-a2b7-d9e7c37d5148 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:00.974583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be62a8-5236-4405-928e-bb6a522de344 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:00.974682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac76081-007e-4d05-a2b7-d9e7c37d5148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:00.974856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be62a8-5236-4405-928e-bb6a522de344 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:00.975123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be62a8-5236-4405-928e-bb6a522de344 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:00.975783 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be62a8-5236-4405-928e-bb6a522de344 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:00.976081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be62a8-5236-4405-928e-bb6a522de344 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:00.977412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be62a8-5236-4405-928e-bb6a522de344 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:00.989208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 545/1078] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:00.993516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-fac76081-007e-4d05-a2b7-d9e7c37d5148 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:00.994754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fac76081-007e-4d05-a2b7-d9e7c37d5148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:01.003792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:01.079969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 534/1079] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:11:01.344501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be62a8-5236-4405-928e-bb6a522de344 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:01.367810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:01.446169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be62a8-5236-4405-928e-bb6a522de344 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:01.479747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b01c2-7c06-4129-b6a6-a8ffd44eab09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:01.480401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b01c2-7c06-4129-b6a6-a8ffd44eab09 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:01.480676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b01c2-7c06-4129-b6a6-a8ffd44eab09 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:01.480945 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b01c2-7c06-4129-b6a6-a8ffd44eab09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:01.493276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b01c2-7c06-4129-b6a6-a8ffd44eab09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-c53b01c2-7c06-4129-b6a6-a8ffd44eab09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:11:01.498637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:01.499747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:11:01.500610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 546/1080] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:11:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:01.505878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac76081-007e-4d05-a2b7-d9e7c37d5148 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:01.602258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac76081-007e-4d05-a2b7-d9e7c37d5148 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:01.634821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2f769-eb1c-43ce-b500-92640ab33336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:01.635414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2f769-eb1c-43ce-b500-92640ab33336 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:01.635722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2f769-eb1c-43ce-b500-92640ab33336 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:01.635997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2f769-eb1c-43ce-b500-92640ab33336 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:01.636645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2f769-eb1c-43ce-b500-92640ab33336 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:01.636943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2f769-eb1c-43ce-b500-92640ab33336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:01.638303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2f769-eb1c-43ce-b500-92640ab33336 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:01.646102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 535/1081] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:11:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:01.651772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b01c2-7c06-4129-b6a6-a8ffd44eab09 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:01.731062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:01.750201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b01c2-7c06-4129-b6a6-a8ffd44eab09 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:01.784023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047b84-c0cf-4c52-9b63-c76eabc5b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:01.784639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047b84-c0cf-4c52-9b63-c76eabc5b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:01.784915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047b84-c0cf-4c52-9b63-c76eabc5b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:01.785182 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047b84-c0cf-4c52-9b63-c76eabc5b3e0 None admin] PATH_INFO: `/v3/users/6c336f83df0145f2ba48dddd05ebd280` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:01.786292 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30047b84-c0cf-4c52-9b63-c76eabc5b3e0 None admin] RBAC: Authorizing `identity:get_user(user_id=6c336f83df0145f2ba48dddd05ebd280)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:01.787741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30047b84-c0cf-4c52-9b63-c76eabc5b3e0 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:01.789309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 547/1082] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:01 2022] GET /identity/v3/users/6c336f83df0145f2ba48dddd05ebd280 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:11:01.794916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2f769-eb1c-43ce-b500-92640ab33336 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:01.798723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:01.891809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2f769-eb1c-43ce-b500-92640ab33336 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:01.924739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b060d-d1f3-4fc6-a23f-bbeb74d4dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:01.925360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b060d-d1f3-4fc6-a23f-bbeb74d4dbdb None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:01.925635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b060d-d1f3-4fc6-a23f-bbeb74d4dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:01.925904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b060d-d1f3-4fc6-a23f-bbeb74d4dbdb None admin] PATH_INFO: `/v3/projects/8fa602f1cad549258cea821337eb25b1` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:01.926995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b060d-d1f3-4fc6-a23f-bbeb74d4dbdb None admin] RBAC: Authorizing `identity:get_project(project_id=8fa602f1cad549258cea821337eb25b1)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:01.928457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b060d-d1f3-4fc6-a23f-bbeb74d4dbdb None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:01.930074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 536/1083] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:01 2022] GET /identity/v3/projects/8fa602f1cad549258cea821337eb25b1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:11:01.935913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047b84-c0cf-4c52-9b63-c76eabc5b3e0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:02.033988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047b84-c0cf-4c52-9b63-c76eabc5b3e0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:02.068240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc8db1-5bc4-4028-bb56-3ff6a15aa8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:02.068910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc8db1-5bc4-4028-bb56-3ff6a15aa8ca None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:02.069191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc8db1-5bc4-4028-bb56-3ff6a15aa8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:02.069463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc8db1-5bc4-4028-bb56-3ff6a15aa8ca None admin] PATH_INFO: `/v3/projects/8fa602f1cad549258cea821337eb25b1/users/6c336f83df0145f2ba48dddd05ebd280/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:02.071338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc8db1-5bc4-4028-bb56-3ff6a15aa8ca None admin] RBAC: Authorizing `identity:create_grant(project_id=8fa602f1cad549258cea821337eb25b1, user_id=6c336f83df0145f2ba48dddd05ebd280, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:02.072776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc8db1-5bc4-4028-bb56-3ff6a15aa8ca None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:02.182187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 548/1084] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:11:01 2022] PUT /identity/v3/projects/8fa602f1cad549258cea821337eb25b1/users/6c336f83df0145f2ba48dddd05ebd280/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:11:02.316584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4590716-8c13-4d14-8e95-7a4bc09fc492 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:02.317119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4590716-8c13-4d14-8e95-7a4bc09fc492 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:02.317119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4590716-8c13-4d14-8e95-7a4bc09fc492 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:02.318772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 537/1085] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:02.324012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc0c0b-09cd-4cca-8abd-3e62df07c637 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:02.324296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc0c0b-09cd-4cca-8abd-3e62df07c637 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:02.324570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc0c0b-09cd-4cca-8abd-3e62df07c637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:02.343871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-57dc0c0b-09cd-4cca-8abd-3e62df07c637 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:02.345103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-57dc0c0b-09cd-4cca-8abd-3e62df07c637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:02.347099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:02.412106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 549/1086] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:02.679326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4590716-8c13-4d14-8e95-7a4bc09fc492 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:02.749673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:02.779298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4590716-8c13-4d14-8e95-7a4bc09fc492 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:02.812971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef51b0e-24c4-49d1-931f-69bbe2968d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:02.813601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef51b0e-24c4-49d1-931f-69bbe2968d52 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:02.813926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef51b0e-24c4-49d1-931f-69bbe2968d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:02.814695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef51b0e-24c4-49d1-931f-69bbe2968d52 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:02.824954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef51b0e-24c4-49d1-931f-69bbe2968d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-8ef51b0e-24c4-49d1-931f-69bbe2968d52 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:11:02.829735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:02.830932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:02.831867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 538/1087] 10.222.0.62 () {56 vars in 1120 bytes} [Mon Sep 5 12:11:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:02.836792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc0c0b-09cd-4cca-8abd-3e62df07c637 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:02.932556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc0c0b-09cd-4cca-8abd-3e62df07c637 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:02.964663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7cb21-a329-4b0f-8fcf-28e12b538592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:02.965254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7cb21-a329-4b0f-8fcf-28e12b538592 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:02.965534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7cb21-a329-4b0f-8fcf-28e12b538592 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:02.965807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7cb21-a329-4b0f-8fcf-28e12b538592 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:02.966462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7cb21-a329-4b0f-8fcf-28e12b538592 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:02.966761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7cb21-a329-4b0f-8fcf-28e12b538592 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:02.968392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7cb21-a329-4b0f-8fcf-28e12b538592 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:02.973814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:02.977013 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 550/1088] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Sep 5 12:11:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:02.982488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef51b0e-24c4-49d1-931f-69bbe2968d52 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:03.079846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef51b0e-24c4-49d1-931f-69bbe2968d52 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:03.109174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:03.114505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9ba50-9b19-4bfb-ad67-d55f85f2f7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:03.115133 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9ba50-9b19-4bfb-ad67-d55f85f2f7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:03.115413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9ba50-9b19-4bfb-ad67-d55f85f2f7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:03.115714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9ba50-9b19-4bfb-ad67-d55f85f2f7d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:03.125897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9ba50-9b19-4bfb-ad67-d55f85f2f7d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:03.127373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9ba50-9b19-4bfb-ad67-d55f85f2f7d9 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-82d9ba50-9b19-4bfb-ad67-d55f85f2f7d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:11:03.141087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:03.142264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:11:03.143424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:11:03.143424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:11:03.143424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:03.146946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 539/1089] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:03.148237 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7cb21-a329-4b0f-8fcf-28e12b538592 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:03.249663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7cb21-a329-4b0f-8fcf-28e12b538592 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:03.282473 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d850651-f705-41e9-b8cb-ab4fb6c48865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:03.283673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d850651-f705-41e9-b8cb-ab4fb6c48865 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:03.283946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d850651-f705-41e9-b8cb-ab4fb6c48865 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:03.284212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d850651-f705-41e9-b8cb-ab4fb6c48865 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:03.284842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d850651-f705-41e9-b8cb-ab4fb6c48865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:03.285138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d850651-f705-41e9-b8cb-ab4fb6c48865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:03.286505 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d850651-f705-41e9-b8cb-ab4fb6c48865 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:03.298782 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 551/1090] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:03.304459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9ba50-9b19-4bfb-ad67-d55f85f2f7d9 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:03.402053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9ba50-9b19-4bfb-ad67-d55f85f2f7d9 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:03.435691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd9810-cef7-4eb8-82ec-c0df1458dd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:03.436310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd9810-cef7-4eb8-82ec-c0df1458dd19 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:03.436583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd9810-cef7-4eb8-82ec-c0df1458dd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:03.436854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd9810-cef7-4eb8-82ec-c0df1458dd19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:03.444087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd9810-cef7-4eb8-82ec-c0df1458dd19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:03.445509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd9810-cef7-4eb8-82ec-c0df1458dd19 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-0ddd9810-cef7-4eb8-82ec-c0df1458dd19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:11:03.456787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:03.457764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:03.458725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 540/1091] 10.222.0.62 () {56 vars in 1093 bytes} [Mon Sep 5 12:11:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:03.463814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d850651-f705-41e9-b8cb-ab4fb6c48865 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:03.521917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:03.561638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d850651-f705-41e9-b8cb-ab4fb6c48865 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:03.596053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffef52-325b-4f34-a469-f51a80f7df9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:03.596658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffef52-325b-4f34-a469-f51a80f7df9d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:03.596932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffef52-325b-4f34-a469-f51a80f7df9d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:03.597200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffef52-325b-4f34-a469-f51a80f7df9d None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:03.597943 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffef52-325b-4f34-a469-f51a80f7df9d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:03.598244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffef52-325b-4f34-a469-f51a80f7df9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:03.599684 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffef52-325b-4f34-a469-f51a80f7df9d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:03.616630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 552/1092] 10.222.0.62 () {56 vars in 1133 bytes} [Mon Sep 5 12:11:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:03.622463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd9810-cef7-4eb8-82ec-c0df1458dd19 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:03.667551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:03.721194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd9810-cef7-4eb8-82ec-c0df1458dd19 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:03.753751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ed6da-fc00-4fbb-afc4-f710f32017e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:03.754348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ed6da-fc00-4fbb-afc4-f710f32017e9 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:03.754624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ed6da-fc00-4fbb-afc4-f710f32017e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:03.754893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ed6da-fc00-4fbb-afc4-f710f32017e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:03.820216 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ed6da-fc00-4fbb-afc4-f710f32017e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:03.821581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ed6da-fc00-4fbb-afc4-f710f32017e9 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:03.864432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:03.917592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 541/1093] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:11:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 12:11:03.923547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffef52-325b-4f34-a469-f51a80f7df9d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:04.023649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffef52-325b-4f34-a469-f51a80f7df9d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:04.041080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:04.051561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf255-cffa-493a-b2ff-ae1cca46a4a2 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.051695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf255-cffa-493a-b2ff-ae1cca46a4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.051910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf255-cffa-493a-b2ff-ae1cca46a4a2 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.053402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 542/1094] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:11:04.058063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb26580-e5e2-4f24-867d-6b9e78b10620 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.058343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb26580-e5e2-4f24-867d-6b9e78b10620 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.058615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb26580-e5e2-4f24-867d-6b9e78b10620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.058732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0cfd8-e937-44fa-b55a-681d52156089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:04.059299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0cfd8-e937-44fa-b55a-681d52156089 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.059597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0cfd8-e937-44fa-b55a-681d52156089 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.059868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0cfd8-e937-44fa-b55a-681d52156089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.071035 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0cfd8-e937-44fa-b55a-681d52156089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:04.074474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0cfd8-e937-44fa-b55a-681d52156089 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:04.077404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-bdb26580-e5e2-4f24-867d-6b9e78b10620 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:04.078612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-bdb26580-e5e2-4f24-867d-6b9e78b10620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-a3b0cfd8-e937-44fa-b55a-681d52156089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:11:04.099159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:04.105143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:11:04.106658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:11:04.106658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:11:04.106658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:11:04.106658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 553/1095] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:04.106658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0cfd8-e937-44fa-b55a-681d52156089 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:04.149000 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 543/1096] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:11:04.202436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0cfd8-e937-44fa-b55a-681d52156089 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:04.235616 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b31b4-b96f-4713-bb20-34a9a74f809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:04.236234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b31b4-b96f-4713-bb20-34a9a74f809b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.236508 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b31b4-b96f-4713-bb20-34a9a74f809b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.236776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b31b4-b96f-4713-bb20-34a9a74f809b None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.237405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b31b4-b96f-4713-bb20-34a9a74f809b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:04.237699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b31b4-b96f-4713-bb20-34a9a74f809b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:04.239045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b31b4-b96f-4713-bb20-34a9a74f809b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:04.250410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 554/1097] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:11:04.256148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb26580-e5e2-4f24-867d-6b9e78b10620 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:04.330606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:04.355855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb26580-e5e2-4f24-867d-6b9e78b10620 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:04.389378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba351f-3057-4be2-b049-685ed75c835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:04.390006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba351f-3057-4be2-b049-685ed75c835a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.390286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba351f-3057-4be2-b049-685ed75c835a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.390557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba351f-3057-4be2-b049-685ed75c835a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.397855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba351f-3057-4be2-b049-685ed75c835a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:04.399298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba351f-3057-4be2-b049-685ed75c835a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:04.418665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b31b4-b96f-4713-bb20-34a9a74f809b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-6aba351f-3057-4be2-b049-685ed75c835a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:11:04.418901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:04.420373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:04.421285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:11:04.421285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:11:04.421285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:11:04.421285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:11:04.421285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:11:04.421285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:11:04.421285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:11:04.421285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:04.421285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 544/1098] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:11:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:04.425865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba351f-3057-4be2-b049-685ed75c835a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:04.450963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:04.522898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b31b4-b96f-4713-bb20-34a9a74f809b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:04.528209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba351f-3057-4be2-b049-685ed75c835a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:04.556311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee23b-850f-4643-aa3c-9f927f615136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:04.556927 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee23b-850f-4643-aa3c-9f927f615136 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.557200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee23b-850f-4643-aa3c-9f927f615136 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.557466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee23b-850f-4643-aa3c-9f927f615136 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.560928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c9329-cab7-4927-afb0-3952478510f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:04.561531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c9329-cab7-4927-afb0-3952478510f8 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.561811 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c9329-cab7-4927-afb0-3952478510f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.562087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c9329-cab7-4927-afb0-3952478510f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.562776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c9329-cab7-4927-afb0-3952478510f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:04.563081 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c9329-cab7-4927-afb0-3952478510f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:04.565051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c9329-cab7-4927-afb0-3952478510f8 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:04.566662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:04.568058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ee23b-850f-4643-aa3c-9f927f615136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-0f7ee23b-850f-4643-aa3c-9f927f615136 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:11:04.572898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:04.573995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:11:04.582480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 555/1099] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:11:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:11:04.583194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 545/1100] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:11:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:11:04.583269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee23b-850f-4643-aa3c-9f927f615136 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:04.584600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c9329-cab7-4927-afb0-3952478510f8 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:04.672839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:04.682100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee23b-850f-4643-aa3c-9f927f615136 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:04.684390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c9329-cab7-4927-afb0-3952478510f8 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:04.716759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca5488-6585-4f22-8a0d-f387e30636cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:04.717355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca5488-6585-4f22-8a0d-f387e30636cd None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.717637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca5488-6585-4f22-8a0d-f387e30636cd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.717906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca5488-6585-4f22-8a0d-f387e30636cd None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.718812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca5488-6585-4f22-8a0d-f387e30636cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:04.719070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343db40a-5c17-43b0-a3a2-3d965e44ab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:04.719201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca5488-6585-4f22-8a0d-f387e30636cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:04.719706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343db40a-5c17-43b0-a3a2-3d965e44ab78 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.720011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343db40a-5c17-43b0-a3a2-3d965e44ab78 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.720288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343db40a-5c17-43b0-a3a2-3d965e44ab78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.720488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca5488-6585-4f22-8a0d-f387e30636cd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:04.721008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343db40a-5c17-43b0-a3a2-3d965e44ab78 None admin] RBAC: Adding query filter params (role.id=8b2491af21004040878bb0b8acb0194d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=7956a7748ec94a618eb466f5536220b0) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:04.721318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343db40a-5c17-43b0-a3a2-3d965e44ab78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:04.722744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343db40a-5c17-43b0-a3a2-3d965e44ab78 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:04.728539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 546/1101] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:11:04 2022] GET /identity/v3/role_assignments?user.id=7956a7748ec94a618eb466f5536220b0&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=8b2491af21004040878bb0b8acb0194d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:04.730395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 556/1102] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:11:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:11:04.736810 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343db40a-5c17-43b0-a3a2-3d965e44ab78 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:04.800882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:04.843970 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343db40a-5c17-43b0-a3a2-3d965e44ab78 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:04.880955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dab31-6eb0-484a-9fb2-a554b67fea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:04.881577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dab31-6eb0-484a-9fb2-a554b67fea46 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:04.881854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dab31-6eb0-484a-9fb2-a554b67fea46 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:04.882123 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dab31-6eb0-484a-9fb2-a554b67fea46 None admin] PATH_INFO: `/v3/users/6c336f83df0145f2ba48dddd05ebd280` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:04.883239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dab31-6eb0-484a-9fb2-a554b67fea46 None admin] RBAC: Authorizing `identity:get_user(user_id=6c336f83df0145f2ba48dddd05ebd280)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:04.884698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dab31-6eb0-484a-9fb2-a554b67fea46 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:04.886267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 547/1103] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:04 2022] GET /identity/v3/users/6c336f83df0145f2ba48dddd05ebd280 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:04.892395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca5488-6585-4f22-8a0d-f387e30636cd None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:04.993290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca5488-6585-4f22-8a0d-f387e30636cd None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:05.028349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd954d-4891-4a39-a97e-d97c4649c4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:05.028950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd954d-4891-4a39-a97e-d97c4649c4eb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:05.029228 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd954d-4891-4a39-a97e-d97c4649c4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:05.029495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd954d-4891-4a39-a97e-d97c4649c4eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:05.051081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:05.101762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd954d-4891-4a39-a97e-d97c4649c4eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:05.103184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd954d-4891-4a39-a97e-d97c4649c4eb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:05.206780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 557/1104] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:11:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:11:05.212569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dab31-6eb0-484a-9fb2-a554b67fea46 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:05.317625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dab31-6eb0-484a-9fb2-a554b67fea46 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:05.342621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:05.351714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322418b-55b4-4b71-b8b9-3ebbfbd3f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:05.352399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322418b-55b4-4b71-b8b9-3ebbfbd3f706 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:05.352625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322418b-55b4-4b71-b8b9-3ebbfbd3f706 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:05.352896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322418b-55b4-4b71-b8b9-3ebbfbd3f706 None admin] PATH_INFO: `/v3/projects/8fa602f1cad549258cea821337eb25b1` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:05.354017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322418b-55b4-4b71-b8b9-3ebbfbd3f706 None admin] RBAC: Authorizing `identity:get_project(project_id=8fa602f1cad549258cea821337eb25b1)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:05.355447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322418b-55b4-4b71-b8b9-3ebbfbd3f706 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:05.357316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 548/1105] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:05 2022] GET /identity/v3/projects/8fa602f1cad549258cea821337eb25b1 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:05.363951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd954d-4891-4a39-a97e-d97c4649c4eb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:05.416777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:05.463246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd954d-4891-4a39-a97e-d97c4649c4eb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:05.497745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9e62d-f36d-455d-a4c4-83354ea2cf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:05.498350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9e62d-f36d-455d-a4c4-83354ea2cf37 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:05.498626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9e62d-f36d-455d-a4c4-83354ea2cf37 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:05.498891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9e62d-f36d-455d-a4c4-83354ea2cf37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:05.499627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9e62d-f36d-455d-a4c4-83354ea2cf37 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=8fa602f1cad549258cea821337eb25b1, user.id=6c336f83df0145f2ba48dddd05ebd280) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:05.499928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9e62d-f36d-455d-a4c4-83354ea2cf37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:05.501321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9e62d-f36d-455d-a4c4-83354ea2cf37 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:05.507123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 558/1106] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:11:05 2022] GET /identity/v3/role_assignments?user.id=6c336f83df0145f2ba48dddd05ebd280&scope.project.id=8fa602f1cad549258cea821337eb25b1&role.id=705907b9bdec47cebf88b868a6a56356 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:07.394903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa18a70-7e8f-4c3e-93bb-46ccfc2b9938 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:07.395405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa18a70-7e8f-4c3e-93bb-46ccfc2b9938 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:07.395405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa18a70-7e8f-4c3e-93bb-46ccfc2b9938 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:07.396843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 549/1107] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:07.401872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092c704-c638-407e-b77d-36e94eb080eb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:07.402263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092c704-c638-407e-b77d-36e94eb080eb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:07.402407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092c704-c638-407e-b77d-36e94eb080eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:07.420299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5092c704-c638-407e-b77d-36e94eb080eb None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:07.421461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5092c704-c638-407e-b77d-36e94eb080eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:07.485175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 559/1108] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:07.756940 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa18a70-7e8f-4c3e-93bb-46ccfc2b9938 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:07.856513 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa18a70-7e8f-4c3e-93bb-46ccfc2b9938 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:07.889985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffaa03d-3f47-477b-b45d-b1fd7d69f0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:07.890599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaa03d-3f47-477b-b45d-b1fd7d69f0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:07.890878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaa03d-3f47-477b-b45d-b1fd7d69f0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:07.891151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaa03d-3f47-477b-b45d-b1fd7d69f0c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:07.892297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffaa03d-3f47-477b-b45d-b1fd7d69f0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:07.893731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffaa03d-3f47-477b-b45d-b1fd7d69f0c4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:07.895293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 550/1109] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:11:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:07.900760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092c704-c638-407e-b77d-36e94eb080eb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:07.946281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:07.997059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092c704-c638-407e-b77d-36e94eb080eb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:08.029398 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc99548-de3e-41b7-bdbc-c7af3fd49e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:08.029987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc99548-de3e-41b7-bdbc-c7af3fd49e5b None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:08.030261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc99548-de3e-41b7-bdbc-c7af3fd49e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:08.030528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc99548-de3e-41b7-bdbc-c7af3fd49e5b None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:08.031164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc99548-de3e-41b7-bdbc-c7af3fd49e5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:08.032592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc99548-de3e-41b7-bdbc-c7af3fd49e5b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:08.102912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 560/1110] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:11:07 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:11:10.025381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ec959-dc46-44de-b0dc-8ca4835facb0 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:10.025910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ec959-dc46-44de-b0dc-8ca4835facb0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:10.025910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ec959-dc46-44de-b0dc-8ca4835facb0 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:10.027238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 551/1111] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:10.032412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68e044-3885-40da-8512-5d5bd93c7b4a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:10.032821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68e044-3885-40da-8512-5d5bd93c7b4a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:10.032952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68e044-3885-40da-8512-5d5bd93c7b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:10.051595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0b68e044-3885-40da-8512-5d5bd93c7b4a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:10.052785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0b68e044-3885-40da-8512-5d5bd93c7b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:10.117282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 561/1112] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:10.382279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ec959-dc46-44de-b0dc-8ca4835facb0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:10.443508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:10.481770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ec959-dc46-44de-b0dc-8ca4835facb0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:10.514809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a42b3b-57bb-47e6-86af-1d89344ce81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:10.515419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a42b3b-57bb-47e6-86af-1d89344ce81a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:10.515729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a42b3b-57bb-47e6-86af-1d89344ce81a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:10.516002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a42b3b-57bb-47e6-86af-1d89344ce81a None admin] PATH_INFO: `/v3/roles/43008618c1dc4cd49109f35c778080d3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:10.517365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a42b3b-57bb-47e6-86af-1d89344ce81a None admin] RBAC: Authorizing `identity:get_role(role_id=43008618c1dc4cd49109f35c778080d3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:10.519333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a42b3b-57bb-47e6-86af-1d89344ce81a None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:10.520649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 552/1113] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:10 2022] GET /identity/v3/roles/43008618c1dc4cd49109f35c778080d3 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:10.526345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68e044-3885-40da-8512-5d5bd93c7b4a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:10.555249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:10.621980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68e044-3885-40da-8512-5d5bd93c7b4a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:10.654808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24734e47-2665-4a11-997d-3ce390849934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:10.655366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734e47-2665-4a11-997d-3ce390849934 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:10.655666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734e47-2665-4a11-997d-3ce390849934 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:10.655934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734e47-2665-4a11-997d-3ce390849934 None admin] PATH_INFO: `/v3/users/5020d32facaf469494762062dc6673c2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:10.668291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24734e47-2665-4a11-997d-3ce390849934 None admin] RBAC: Authorizing `identity:get_user(user_id=5020d32facaf469494762062dc6673c2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:10.669707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24734e47-2665-4a11-997d-3ce390849934 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:10.671244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 562/1114] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:10 2022] GET /identity/v3/users/5020d32facaf469494762062dc6673c2 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:10.676851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a42b3b-57bb-47e6-86af-1d89344ce81a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:10.774665 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a42b3b-57bb-47e6-86af-1d89344ce81a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:10.802245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:10.807762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2205552-b1d8-4175-ab23-48181b1920d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:10.808365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2205552-b1d8-4175-ab23-48181b1920d6 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:10.808644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2205552-b1d8-4175-ab23-48181b1920d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:10.808917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2205552-b1d8-4175-ab23-48181b1920d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:10.873617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2205552-b1d8-4175-ab23-48181b1920d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:10.875052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2205552-b1d8-4175-ab23-48181b1920d6 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:10.970215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 553/1115] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:11:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:11:10.976045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24734e47-2665-4a11-997d-3ce390849934 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:11.062105 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:11.071928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24734e47-2665-4a11-997d-3ce390849934 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:11.105014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39dfc0-55b4-4522-a660-25f64b65bf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:11.105631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39dfc0-55b4-4522-a660-25f64b65bf98 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:11.105905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39dfc0-55b4-4522-a660-25f64b65bf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:11.106173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39dfc0-55b4-4522-a660-25f64b65bf98 None admin] PATH_INFO: `/v3/projects/8fa602f1cad549258cea821337eb25b1` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:11.107270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39dfc0-55b4-4522-a660-25f64b65bf98 None admin] RBAC: Authorizing `identity:get_project(project_id=8fa602f1cad549258cea821337eb25b1)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:11.108711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39dfc0-55b4-4522-a660-25f64b65bf98 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:11.110311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 563/1116] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:10 2022] GET /identity/v3/projects/8fa602f1cad549258cea821337eb25b1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:11.116907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2205552-b1d8-4175-ab23-48181b1920d6 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:11.146074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:11.213180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2205552-b1d8-4175-ab23-48181b1920d6 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:11.246618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee4fed-d80c-4bf0-b0a2-6a8c4d002679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:11.247151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee4fed-d80c-4bf0-b0a2-6a8c4d002679 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:11.247429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee4fed-d80c-4bf0-b0a2-6a8c4d002679 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:11.247897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee4fed-d80c-4bf0-b0a2-6a8c4d002679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:11.248610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee4fed-d80c-4bf0-b0a2-6a8c4d002679 None admin] RBAC: Adding query filter params (role.id=43008618c1dc4cd49109f35c778080d3, scope.project.id=8fa602f1cad549258cea821337eb25b1, user.id=5020d32facaf469494762062dc6673c2) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:11.248916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee4fed-d80c-4bf0-b0a2-6a8c4d002679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:11.250333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee4fed-d80c-4bf0-b0a2-6a8c4d002679 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:11.255516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 554/1117] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:11:11 2022] GET /identity/v3/role_assignments?user.id=5020d32facaf469494762062dc6673c2&scope.project.id=8fa602f1cad549258cea821337eb25b1&role.id=43008618c1dc4cd49109f35c778080d3 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:13.101564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adae3-5574-453f-a249-5ffc3b65abe6 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:13.102054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adae3-5574-453f-a249-5ffc3b65abe6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:13.102154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adae3-5574-453f-a249-5ffc3b65abe6 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:13.103403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 564/1118] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:13.108412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88292a0c-d3f1-4e2c-ac69-1ca2ce930776 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:13.108816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88292a0c-d3f1-4e2c-ac69-1ca2ce930776 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:13.108953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88292a0c-d3f1-4e2c-ac69-1ca2ce930776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:13.126767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-88292a0c-d3f1-4e2c-ac69-1ca2ce930776 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:13.127993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-88292a0c-d3f1-4e2c-ac69-1ca2ce930776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:13.193026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 555/1119] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:13.460678 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29adae3-5574-453f-a249-5ffc3b65abe6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:13.557755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29adae3-5574-453f-a249-5ffc3b65abe6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:13.590524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9790fb-7776-49d4-a02c-54b556cecaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:13.591126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9790fb-7776-49d4-a02c-54b556cecaaa None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:13.591399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9790fb-7776-49d4-a02c-54b556cecaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:13.591699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9790fb-7776-49d4-a02c-54b556cecaaa None admin] PATH_INFO: `/v3/roles/43008618c1dc4cd49109f35c778080d3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:13.593089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9790fb-7776-49d4-a02c-54b556cecaaa None admin] RBAC: Authorizing `identity:get_role(role_id=43008618c1dc4cd49109f35c778080d3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:13.594653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9790fb-7776-49d4-a02c-54b556cecaaa None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:13.595944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 565/1120] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:13 2022] GET /identity/v3/roles/43008618c1dc4cd49109f35c778080d3 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:13.601486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88292a0c-d3f1-4e2c-ac69-1ca2ce930776 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:13.625127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:13.696385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88292a0c-d3f1-4e2c-ac69-1ca2ce930776 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:13.729602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df32522-6d9a-4300-b7f2-a81868da2380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:13.730163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df32522-6d9a-4300-b7f2-a81868da2380 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:13.730439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df32522-6d9a-4300-b7f2-a81868da2380 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:13.730709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df32522-6d9a-4300-b7f2-a81868da2380 None admin] PATH_INFO: `/v3/users/5020d32facaf469494762062dc6673c2` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:13.731831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df32522-6d9a-4300-b7f2-a81868da2380 None admin] RBAC: Authorizing `identity:get_user(user_id=5020d32facaf469494762062dc6673c2)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:13.733248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df32522-6d9a-4300-b7f2-a81868da2380 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:13.734790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 556/1121] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:13 2022] GET /identity/v3/users/5020d32facaf469494762062dc6673c2 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:13.740852 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9790fb-7776-49d4-a02c-54b556cecaaa None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:13.767732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:13.838526 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9790fb-7776-49d4-a02c-54b556cecaaa None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:13.873246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303eadf-c795-4975-8bd0-e51b9b88abb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:13.873858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303eadf-c795-4975-8bd0-e51b9b88abb4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:13.874136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303eadf-c795-4975-8bd0-e51b9b88abb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:13.874409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303eadf-c795-4975-8bd0-e51b9b88abb4 None admin] PATH_INFO: `/v3/projects/8fa602f1cad549258cea821337eb25b1` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:13.875513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303eadf-c795-4975-8bd0-e51b9b88abb4 None admin] RBAC: Authorizing `identity:get_project(project_id=8fa602f1cad549258cea821337eb25b1)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:13.876952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303eadf-c795-4975-8bd0-e51b9b88abb4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:13.879167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 566/1122] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:13 2022] GET /identity/v3/projects/8fa602f1cad549258cea821337eb25b1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:13.884799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df32522-6d9a-4300-b7f2-a81868da2380 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:13.973178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:13.985713 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df32522-6d9a-4300-b7f2-a81868da2380 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:14.015282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547c5b5-fd4a-4c9c-b6cf-e816a35c31da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:14.015959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547c5b5-fd4a-4c9c-b6cf-e816a35c31da None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:14.016235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547c5b5-fd4a-4c9c-b6cf-e816a35c31da None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:14.016510 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547c5b5-fd4a-4c9c-b6cf-e816a35c31da None admin] PATH_INFO: `/v3/projects/8fa602f1cad549258cea821337eb25b1/users/5020d32facaf469494762062dc6673c2/roles/43008618c1dc4cd49109f35c778080d3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:14.018645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547c5b5-fd4a-4c9c-b6cf-e816a35c31da None admin] RBAC: Authorizing `identity:create_grant(project_id=8fa602f1cad549258cea821337eb25b1, user_id=5020d32facaf469494762062dc6673c2, role_id=43008618c1dc4cd49109f35c778080d3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:14.020097 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547c5b5-fd4a-4c9c-b6cf-e816a35c31da None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:14.108213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 557/1123] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:11:13 2022] PUT /identity/v3/projects/8fa602f1cad549258cea821337eb25b1/users/5020d32facaf469494762062dc6673c2/roles/43008618c1dc4cd49109f35c778080d3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:11:15.956896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c24e41-9210-41ae-9a75-18235edce4c2 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:15.957487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c24e41-9210-41ae-9a75-18235edce4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:15.957487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c24e41-9210-41ae-9a75-18235edce4c2 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:15.958737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 567/1124] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:15.963833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d692a-2673-4e06-a85c-ccd658952ca4 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:15.964294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d692a-2673-4e06-a85c-ccd658952ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:15.964407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d692a-2673-4e06-a85c-ccd658952ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:15.982850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-9a0d692a-2673-4e06-a85c-ccd658952ca4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:15.984094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9a0d692a-2673-4e06-a85c-ccd658952ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:16.048294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 558/1125] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 12:11:16.312453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c24e41-9210-41ae-9a75-18235edce4c2 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:16.403799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:16.411022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c24e41-9210-41ae-9a75-18235edce4c2 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:16.442994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b241720-8c13-40b6-9c36-311f5a58d4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:16.443639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b241720-8c13-40b6-9c36-311f5a58d4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:16.443916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b241720-8c13-40b6-9c36-311f5a58d4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:16.444186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b241720-8c13-40b6-9c36-311f5a58d4d6 None admin] PATH_INFO: `/v3/roles/43008618c1dc4cd49109f35c778080d3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:16.445704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b241720-8c13-40b6-9c36-311f5a58d4d6 None admin] RBAC: Authorizing `identity:get_role(role_id=43008618c1dc4cd49109f35c778080d3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:16.447644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b241720-8c13-40b6-9c36-311f5a58d4d6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:16.449073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 568/1126] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:16 2022] GET /identity/v3/roles/43008618c1dc4cd49109f35c778080d3 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:16.454677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d692a-2673-4e06-a85c-ccd658952ca4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:16.533691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:16.550697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d692a-2673-4e06-a85c-ccd658952ca4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:16.583952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2e5ae-e2b6-4831-8303-56b64b77b5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:16.584513 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2e5ae-e2b6-4831-8303-56b64b77b5bf None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:16.584790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2e5ae-e2b6-4831-8303-56b64b77b5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:16.585069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2e5ae-e2b6-4831-8303-56b64b77b5bf None admin] PATH_INFO: `/v3/users/5020d32facaf469494762062dc6673c2` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:16.586169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2e5ae-e2b6-4831-8303-56b64b77b5bf None admin] RBAC: Authorizing `identity:get_user(user_id=5020d32facaf469494762062dc6673c2)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:16.587619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2e5ae-e2b6-4831-8303-56b64b77b5bf None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:16.589164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 559/1127] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:16 2022] GET /identity/v3/users/5020d32facaf469494762062dc6673c2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:16.594976 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b241720-8c13-40b6-9c36-311f5a58d4d6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:16.689382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b241720-8c13-40b6-9c36-311f5a58d4d6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:16.722442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcde72d-3e7a-4086-9e5e-08ad942d6592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:16.723026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde72d-3e7a-4086-9e5e-08ad942d6592 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:16.723299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde72d-3e7a-4086-9e5e-08ad942d6592 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:16.723585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde72d-3e7a-4086-9e5e-08ad942d6592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:16.751864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:16.789645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcde72d-3e7a-4086-9e5e-08ad942d6592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:16.791047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcde72d-3e7a-4086-9e5e-08ad942d6592 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:16.884936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 569/1128] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:11:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:11:16.890691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2e5ae-e2b6-4831-8303-56b64b77b5bf None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:16.986712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2e5ae-e2b6-4831-8303-56b64b77b5bf None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:17.020062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bfc2c-b7d5-47a5-9cf5-cebeddbfebc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:17.020671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bfc2c-b7d5-47a5-9cf5-cebeddbfebc1 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:17.020945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bfc2c-b7d5-47a5-9cf5-cebeddbfebc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:17.021214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bfc2c-b7d5-47a5-9cf5-cebeddbfebc1 None admin] PATH_INFO: `/v3/projects/8fa602f1cad549258cea821337eb25b1` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:17.022345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bfc2c-b7d5-47a5-9cf5-cebeddbfebc1 None admin] RBAC: Authorizing `identity:get_project(project_id=8fa602f1cad549258cea821337eb25b1)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:17.023789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bfc2c-b7d5-47a5-9cf5-cebeddbfebc1 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:17.025371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 560/1129] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:16 2022] GET /identity/v3/projects/8fa602f1cad549258cea821337eb25b1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:17.031369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcde72d-3e7a-4086-9e5e-08ad942d6592 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:17.090216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:17.128050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcde72d-3e7a-4086-9e5e-08ad942d6592 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:17.160449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c015b-469f-426f-80bd-d444827e5a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:17.160983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c015b-469f-426f-80bd-d444827e5a20 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:17.161256 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c015b-469f-426f-80bd-d444827e5a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:17.161525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c015b-469f-426f-80bd-d444827e5a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:17.162233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c015b-469f-426f-80bd-d444827e5a20 None admin] RBAC: Adding query filter params (role.id=43008618c1dc4cd49109f35c778080d3, scope.project.id=8fa602f1cad549258cea821337eb25b1, user.id=5020d32facaf469494762062dc6673c2) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:17.162534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c015b-469f-426f-80bd-d444827e5a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:17.163966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c015b-469f-426f-80bd-d444827e5a20 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:17.169459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 570/1130] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:11:17 2022] GET /identity/v3/role_assignments?user.id=5020d32facaf469494762062dc6673c2&scope.project.id=8fa602f1cad549258cea821337eb25b1&role.id=43008618c1dc4cd49109f35c778080d3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:19.094591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb6306-fff4-4640-9e72-f11aa424db7c None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:19.095326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb6306-fff4-4640-9e72-f11aa424db7c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:19.095430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb6306-fff4-4640-9e72-f11aa424db7c None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:19.096785 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 561/1131] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:19.102627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238b163-e95e-4b3b-8169-c892bf845daf None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:19.103015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238b163-e95e-4b3b-8169-c892bf845daf None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:19.103161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238b163-e95e-4b3b-8169-c892bf845daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:19.120796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0238b163-e95e-4b3b-8169-c892bf845daf None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:19.121964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0238b163-e95e-4b3b-8169-c892bf845daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:19.187808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 571/1132] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:19.457041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb6306-fff4-4640-9e72-f11aa424db7c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:19.477156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:19.559722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb6306-fff4-4640-9e72-f11aa424db7c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:19.592877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc036f69-2ad4-4dd4-9447-fce4a46a1095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:19.593492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc036f69-2ad4-4dd4-9447-fce4a46a1095 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:19.593769 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc036f69-2ad4-4dd4-9447-fce4a46a1095 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:19.594038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc036f69-2ad4-4dd4-9447-fce4a46a1095 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:19.595383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc036f69-2ad4-4dd4-9447-fce4a46a1095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:19.596835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc036f69-2ad4-4dd4-9447-fce4a46a1095 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:19.598404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 562/1133] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:11:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:19.604211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238b163-e95e-4b3b-8169-c892bf845daf None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:19.658566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:19.700155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238b163-e95e-4b3b-8169-c892bf845daf None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:19.736724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366844a-60d5-414b-82d9-51e9c9602623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:19.737250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366844a-60d5-414b-82d9-51e9c9602623 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:19.737525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366844a-60d5-414b-82d9-51e9c9602623 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:19.737794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366844a-60d5-414b-82d9-51e9c9602623 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:19.738441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366844a-60d5-414b-82d9-51e9c9602623 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:19.740412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366844a-60d5-414b-82d9-51e9c9602623 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:19.834533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 572/1134] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:11:19 2022] POST /identity/v3/projects => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:11:21.757001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec952e-0b1d-48fa-991c-e7d8b8c5269c None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:21.757551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec952e-0b1d-48fa-991c-e7d8b8c5269c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:21.757551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec952e-0b1d-48fa-991c-e7d8b8c5269c None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:21.758851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 563/1135] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:21.763956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f728f7-7aab-4db1-b714-f1b01801c205 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:21.764389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f728f7-7aab-4db1-b714-f1b01801c205 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:21.764482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f728f7-7aab-4db1-b714-f1b01801c205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:21.782949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-11f728f7-7aab-4db1-b714-f1b01801c205 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:21.784165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-11f728f7-7aab-4db1-b714-f1b01801c205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:21.848390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 573/1136] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:22.115243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec952e-0b1d-48fa-991c-e7d8b8c5269c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:22.199074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:22.214317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec952e-0b1d-48fa-991c-e7d8b8c5269c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:22.247342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6b23e-b3df-4386-9bec-2e40bae92433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:22.247981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b23e-b3df-4386-9bec-2e40bae92433 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:22.248257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b23e-b3df-4386-9bec-2e40bae92433 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:22.248525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b23e-b3df-4386-9bec-2e40bae92433 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:22.249607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6b23e-b3df-4386-9bec-2e40bae92433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:22.251023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6b23e-b3df-4386-9bec-2e40bae92433 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:22.252584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 564/1137] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:11:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:22.258944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f728f7-7aab-4db1-b714-f1b01801c205 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:22.296730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:22.355284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f728f7-7aab-4db1-b714-f1b01801c205 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:22.387958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720bbdc-330b-48df-97c9-e05f0a0f1fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:22.388552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720bbdc-330b-48df-97c9-e05f0a0f1fdd None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:22.388827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720bbdc-330b-48df-97c9-e05f0a0f1fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:22.389100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720bbdc-330b-48df-97c9-e05f0a0f1fdd None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:22.389742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720bbdc-330b-48df-97c9-e05f0a0f1fdd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:22.391149 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720bbdc-330b-48df-97c9-e05f0a0f1fdd None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:22.496210 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 574/1138] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:11:22 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:11:24.419207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eda1c8-795d-4310-96b0-445ac8719b53 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:24.419749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eda1c8-795d-4310-96b0-445ac8719b53 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:24.419749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eda1c8-795d-4310-96b0-445ac8719b53 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:24.421035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 565/1139] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:24.426254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049685-1453-4e09-892d-ac5343c93e60 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:24.426696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049685-1453-4e09-892d-ac5343c93e60 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:24.426827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049685-1453-4e09-892d-ac5343c93e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:24.444758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2c049685-1453-4e09-892d-ac5343c93e60 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:24.445933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2c049685-1453-4e09-892d-ac5343c93e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:24.510474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 575/1140] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:24.779681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eda1c8-795d-4310-96b0-445ac8719b53 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:24.878305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eda1c8-795d-4310-96b0-445ac8719b53 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:24.910122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b293c9-c7d5-4615-aa44-3d48604150a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:24.911236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b293c9-c7d5-4615-aa44-3d48604150a0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:24.911520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b293c9-c7d5-4615-aa44-3d48604150a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:24.911819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b293c9-c7d5-4615-aa44-3d48604150a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:24.921464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b293c9-c7d5-4615-aa44-3d48604150a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-d8b293c9-c7d5-4615-aa44-3d48604150a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:11:24.926161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:24.927916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:24.929024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 566/1141] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:11:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:24.933638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c049685-1453-4e09-892d-ac5343c93e60 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:24.957229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:25.029980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c049685-1453-4e09-892d-ac5343c93e60 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:25.064563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9fd0e-9c3f-47b5-9187-2c34b902f9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:25.065161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9fd0e-9c3f-47b5-9187-2c34b902f9ac None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:25.065437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9fd0e-9c3f-47b5-9187-2c34b902f9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:25.065713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9fd0e-9c3f-47b5-9187-2c34b902f9ac None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:25.066369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9fd0e-9c3f-47b5-9187-2c34b902f9ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:25.066667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9fd0e-9c3f-47b5-9187-2c34b902f9ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:25.068095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9fd0e-9c3f-47b5-9187-2c34b902f9ac None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:25.076039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 576/1142] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:11:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:25.081809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b293c9-c7d5-4615-aa44-3d48604150a0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:25.099169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:25.181909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b293c9-c7d5-4615-aa44-3d48604150a0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:25.215888 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c1655-7056-4be9-87b7-25e9a93db80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:25.216504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c1655-7056-4be9-87b7-25e9a93db80e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:25.216784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c1655-7056-4be9-87b7-25e9a93db80e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:25.217056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c1655-7056-4be9-87b7-25e9a93db80e None admin] PATH_INFO: `/v3/users/a04a5999ff0247c9a39ae9af9b1a6f55` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:25.230473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c1655-7056-4be9-87b7-25e9a93db80e None admin] RBAC: Authorizing `identity:get_user(user_id=a04a5999ff0247c9a39ae9af9b1a6f55)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:25.231948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c1655-7056-4be9-87b7-25e9a93db80e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:25.233524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 567/1143] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:25 2022] GET /identity/v3/users/a04a5999ff0247c9a39ae9af9b1a6f55 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:25.239428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9fd0e-9c3f-47b5-9187-2c34b902f9ac None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:25.321691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:25.336435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9fd0e-9c3f-47b5-9187-2c34b902f9ac None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:25.368921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4368f-f88b-476e-a074-e61d902ea198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:25.369514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4368f-f88b-476e-a074-e61d902ea198 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:25.369793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4368f-f88b-476e-a074-e61d902ea198 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:25.370067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4368f-f88b-476e-a074-e61d902ea198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:25.436068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4368f-f88b-476e-a074-e61d902ea198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:25.437486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4368f-f88b-476e-a074-e61d902ea198 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:25.578174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:25.593843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 577/1144] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:11:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:11:25.600328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c1655-7056-4be9-87b7-25e9a93db80e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:25.668584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:25.697183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c1655-7056-4be9-87b7-25e9a93db80e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:25.730050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40435c-0c57-4978-8385-2374941a3d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:25.730657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40435c-0c57-4978-8385-2374941a3d03 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:25.730931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40435c-0c57-4978-8385-2374941a3d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:25.731199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40435c-0c57-4978-8385-2374941a3d03 None admin] PATH_INFO: `/v3/projects/6c1d6a9defc340608e6d9cdea4b5b302` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:25.732307 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40435c-0c57-4978-8385-2374941a3d03 None admin] RBAC: Authorizing `identity:get_project(project_id=6c1d6a9defc340608e6d9cdea4b5b302)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:25.733727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40435c-0c57-4978-8385-2374941a3d03 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:25.735315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 568/1145] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:25 2022] GET /identity/v3/projects/6c1d6a9defc340608e6d9cdea4b5b302 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:25.741775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4368f-f88b-476e-a074-e61d902ea198 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:25.836351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4368f-f88b-476e-a074-e61d902ea198 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:25.869278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca52cc-9d99-403c-ba2c-52b9d9a245ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:25.869859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca52cc-9d99-403c-ba2c-52b9d9a245ed None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:25.870131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca52cc-9d99-403c-ba2c-52b9d9a245ed None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:25.870402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca52cc-9d99-403c-ba2c-52b9d9a245ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:25.871109 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca52cc-9d99-403c-ba2c-52b9d9a245ed None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=6c1d6a9defc340608e6d9cdea4b5b302, user.id=a04a5999ff0247c9a39ae9af9b1a6f55) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:25.871408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca52cc-9d99-403c-ba2c-52b9d9a245ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:25.872841 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca52cc-9d99-403c-ba2c-52b9d9a245ed None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:25.877937 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 578/1146] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:11:25 2022] GET /identity/v3/role_assignments?user.id=a04a5999ff0247c9a39ae9af9b1a6f55&scope.project.id=6c1d6a9defc340608e6d9cdea4b5b302&role.id=705907b9bdec47cebf88b868a6a56356 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:27.715453 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60a249-ff57-4ccf-b1c2-92fb482984bb None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:27.716030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60a249-ff57-4ccf-b1c2-92fb482984bb None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:27.716030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60a249-ff57-4ccf-b1c2-92fb482984bb None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:27.717342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 569/1147] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:27.722592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac4c7e-e3dc-44cd-a6ac-0a3de9e49637 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:27.723033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac4c7e-e3dc-44cd-a6ac-0a3de9e49637 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:27.723138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac4c7e-e3dc-44cd-a6ac-0a3de9e49637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:27.741813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f9ac4c7e-e3dc-44cd-a6ac-0a3de9e49637 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:27.742978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f9ac4c7e-e3dc-44cd-a6ac-0a3de9e49637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:27.786947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:27.806936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 579/1148] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:28.072701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60a249-ff57-4ccf-b1c2-92fb482984bb None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:28.171182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60a249-ff57-4ccf-b1c2-92fb482984bb None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:28.204329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece69867-18ab-44a5-8835-febc97fb24f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:28.204942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece69867-18ab-44a5-8835-febc97fb24f3 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:28.205217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece69867-18ab-44a5-8835-febc97fb24f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:28.205483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece69867-18ab-44a5-8835-febc97fb24f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:28.212810 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:28.215750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece69867-18ab-44a5-8835-febc97fb24f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-ece69867-18ab-44a5-8835-febc97fb24f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:11:28.220449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:28.221849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:28.223042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 570/1149] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:11:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:28.227751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac4c7e-e3dc-44cd-a6ac-0a3de9e49637 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:28.324294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac4c7e-e3dc-44cd-a6ac-0a3de9e49637 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:28.356870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8bf70-f26a-4418-bd0d-c56e5440853e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:28.357460 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8bf70-f26a-4418-bd0d-c56e5440853e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:28.357734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8bf70-f26a-4418-bd0d-c56e5440853e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:28.358002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8bf70-f26a-4418-bd0d-c56e5440853e None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:28.358652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8bf70-f26a-4418-bd0d-c56e5440853e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:28.358946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8bf70-f26a-4418-bd0d-c56e5440853e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:28.360314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8bf70-f26a-4418-bd0d-c56e5440853e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:28.368646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 580/1150] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:11:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:28.374358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece69867-18ab-44a5-8835-febc97fb24f3 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:28.471212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece69867-18ab-44a5-8835-febc97fb24f3 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:28.504809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa439e-c004-40f5-85ff-3cbb3da02549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:28.505447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa439e-c004-40f5-85ff-3cbb3da02549 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:28.505726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa439e-c004-40f5-85ff-3cbb3da02549 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:28.506000 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa439e-c004-40f5-85ff-3cbb3da02549 None admin] PATH_INFO: `/v3/users/a04a5999ff0247c9a39ae9af9b1a6f55` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:28.507102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa439e-c004-40f5-85ff-3cbb3da02549 None admin] RBAC: Authorizing `identity:get_user(user_id=a04a5999ff0247c9a39ae9af9b1a6f55)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:28.508555 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa439e-c004-40f5-85ff-3cbb3da02549 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:28.510103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 571/1151] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:28 2022] GET /identity/v3/users/a04a5999ff0247c9a39ae9af9b1a6f55 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:28.515738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8bf70-f26a-4418-bd0d-c56e5440853e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:28.558077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:28.613383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8bf70-f26a-4418-bd0d-c56e5440853e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:28.645282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a8c34-3a9e-407e-bf5d-042176dc8af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:28.646477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a8c34-3a9e-407e-bf5d-042176dc8af8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:28.646752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a8c34-3a9e-407e-bf5d-042176dc8af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:28.647016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a8c34-3a9e-407e-bf5d-042176dc8af8 None admin] PATH_INFO: `/v3/projects/6c1d6a9defc340608e6d9cdea4b5b302` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:28.648127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a8c34-3a9e-407e-bf5d-042176dc8af8 None admin] RBAC: Authorizing `identity:get_project(project_id=6c1d6a9defc340608e6d9cdea4b5b302)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:28.649517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a8c34-3a9e-407e-bf5d-042176dc8af8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:28.651085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 581/1152] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:28 2022] GET /identity/v3/projects/6c1d6a9defc340608e6d9cdea4b5b302 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:28.656898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa439e-c004-40f5-85ff-3cbb3da02549 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:28.722001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:28.754194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa439e-c004-40f5-85ff-3cbb3da02549 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:28.788006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09330-104f-47d9-a744-128a55c019ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:28.788643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09330-104f-47d9-a744-128a55c019ed None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:28.788918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09330-104f-47d9-a744-128a55c019ed None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:28.789189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09330-104f-47d9-a744-128a55c019ed None admin] PATH_INFO: `/v3/projects/6c1d6a9defc340608e6d9cdea4b5b302/users/a04a5999ff0247c9a39ae9af9b1a6f55/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:28.791036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d09330-104f-47d9-a744-128a55c019ed None admin] RBAC: Authorizing `identity:create_grant(project_id=6c1d6a9defc340608e6d9cdea4b5b302, user_id=a04a5999ff0247c9a39ae9af9b1a6f55, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:28.792474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d09330-104f-47d9-a744-128a55c019ed None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:28.878585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 572/1153] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:11:28 2022] PUT /identity/v3/projects/6c1d6a9defc340608e6d9cdea4b5b302/users/a04a5999ff0247c9a39ae9af9b1a6f55/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:11:30.731124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038f932-1ec8-4c0d-8eab-a3c0e1161d21 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:30.731677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038f932-1ec8-4c0d-8eab-a3c0e1161d21 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:30.731677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038f932-1ec8-4c0d-8eab-a3c0e1161d21 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:30.733008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 582/1154] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:30.738187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85f86d-5a50-477b-b62d-bfb42e2a85ee None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:30.738605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85f86d-5a50-477b-b62d-bfb42e2a85ee None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:30.738740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85f86d-5a50-477b-b62d-bfb42e2a85ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:30.757067 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ad85f86d-5a50-477b-b62d-bfb42e2a85ee None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:30.758236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ad85f86d-5a50-477b-b62d-bfb42e2a85ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:30.822992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 573/1155] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:31.091434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038f932-1ec8-4c0d-8eab-a3c0e1161d21 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:31.190254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038f932-1ec8-4c0d-8eab-a3c0e1161d21 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:31.200643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:31.223171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c3697-22be-4201-a932-ba1bc811dbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:31.223837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c3697-22be-4201-a932-ba1bc811dbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:31.224119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c3697-22be-4201-a932-ba1bc811dbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:31.225203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c3697-22be-4201-a932-ba1bc811dbb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:31.234635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c3697-22be-4201-a932-ba1bc811dbb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-a24c3697-22be-4201-a932-ba1bc811dbb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:11:31.239907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:31.241024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:11:31.242281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 583/1156] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:11:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:31.246987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85f86d-5a50-477b-b62d-bfb42e2a85ee None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:31.302809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:31.342509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85f86d-5a50-477b-b62d-bfb42e2a85ee None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:31.376039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82f728-bbb7-49ca-9976-54fefdfcf80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:31.376627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82f728-bbb7-49ca-9976-54fefdfcf80d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:31.376901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82f728-bbb7-49ca-9976-54fefdfcf80d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:31.377170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82f728-bbb7-49ca-9976-54fefdfcf80d None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:31.377814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82f728-bbb7-49ca-9976-54fefdfcf80d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:31.378110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82f728-bbb7-49ca-9976-54fefdfcf80d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:31.379460 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82f728-bbb7-49ca-9976-54fefdfcf80d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:31.387171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 574/1157] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:11:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:31.392842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c3697-22be-4201-a932-ba1bc811dbb6 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:31.488331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c3697-22be-4201-a932-ba1bc811dbb6 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:31.520225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53df254-2e8e-4aac-9884-904aca57582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:31.520857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53df254-2e8e-4aac-9884-904aca57582f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:31.521128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53df254-2e8e-4aac-9884-904aca57582f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:31.521397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53df254-2e8e-4aac-9884-904aca57582f None admin] PATH_INFO: `/v3/users/a04a5999ff0247c9a39ae9af9b1a6f55` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:31.522486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53df254-2e8e-4aac-9884-904aca57582f None admin] RBAC: Authorizing `identity:get_user(user_id=a04a5999ff0247c9a39ae9af9b1a6f55)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:31.523915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53df254-2e8e-4aac-9884-904aca57582f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:31.525453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 584/1158] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:31 2022] GET /identity/v3/users/a04a5999ff0247c9a39ae9af9b1a6f55 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:31.531165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82f728-bbb7-49ca-9976-54fefdfcf80d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:31.628254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82f728-bbb7-49ca-9976-54fefdfcf80d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:31.646873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:31.660185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69f0d7-3683-4553-bdf3-1742067a6802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:31.660782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f0d7-3683-4553-bdf3-1742067a6802 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:31.661059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f0d7-3683-4553-bdf3-1742067a6802 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:31.661840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f0d7-3683-4553-bdf3-1742067a6802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:31.727046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69f0d7-3683-4553-bdf3-1742067a6802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:31.728500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69f0d7-3683-4553-bdf3-1742067a6802 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:31.821568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 575/1159] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:11:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:11:31.827278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53df254-2e8e-4aac-9884-904aca57582f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:31.866627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:31.924979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53df254-2e8e-4aac-9884-904aca57582f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:31.956625 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45478d52-b157-4458-9d47-d7adf62a2e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:31.957243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45478d52-b157-4458-9d47-d7adf62a2e47 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:31.957521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45478d52-b157-4458-9d47-d7adf62a2e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:31.957793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45478d52-b157-4458-9d47-d7adf62a2e47 None admin] PATH_INFO: `/v3/projects/6c1d6a9defc340608e6d9cdea4b5b302` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:31.959414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45478d52-b157-4458-9d47-d7adf62a2e47 None admin] RBAC: Authorizing `identity:get_project(project_id=6c1d6a9defc340608e6d9cdea4b5b302)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:31.960850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45478d52-b157-4458-9d47-d7adf62a2e47 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:31.962435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 585/1160] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:31 2022] GET /identity/v3/projects/6c1d6a9defc340608e6d9cdea4b5b302 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:31.968317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69f0d7-3683-4553-bdf3-1742067a6802 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:31.989531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:32.064352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69f0d7-3683-4553-bdf3-1742067a6802 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:32.098199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5f85f-9a0a-4369-8d46-11b36d18c6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:32.098822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5f85f-9a0a-4369-8d46-11b36d18c6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:32.099102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5f85f-9a0a-4369-8d46-11b36d18c6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:32.099375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5f85f-9a0a-4369-8d46-11b36d18c6e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:32.100115 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5f85f-9a0a-4369-8d46-11b36d18c6e2 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=6c1d6a9defc340608e6d9cdea4b5b302, user.id=a04a5999ff0247c9a39ae9af9b1a6f55) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:32.100421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5f85f-9a0a-4369-8d46-11b36d18c6e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:32.101843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5f85f-9a0a-4369-8d46-11b36d18c6e2 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:32.107424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 576/1161] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:11:31 2022] GET /identity/v3/role_assignments?user.id=a04a5999ff0247c9a39ae9af9b1a6f55&scope.project.id=6c1d6a9defc340608e6d9cdea4b5b302&role.id=705907b9bdec47cebf88b868a6a56356 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:33.998683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe040-f38c-45ef-a417-56f739cd5c75 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:33.999229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe040-f38c-45ef-a417-56f739cd5c75 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:33.999229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe040-f38c-45ef-a417-56f739cd5c75 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:34.000611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 586/1162] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:34.006225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceceb4d-8f35-49db-a9c0-c8cd3eca8c88 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:34.006617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceceb4d-8f35-49db-a9c0-c8cd3eca8c88 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:34.006766 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceceb4d-8f35-49db-a9c0-c8cd3eca8c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:34.024522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-fceceb4d-8f35-49db-a9c0-c8cd3eca8c88 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:34.025696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fceceb4d-8f35-49db-a9c0-c8cd3eca8c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:34.090716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 577/1163] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:34.359199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fe040-f38c-45ef-a417-56f739cd5c75 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:34.458138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fe040-f38c-45ef-a417-56f739cd5c75 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:34.464686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:34.491083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50ac3c-cff7-465e-a862-e0a41bc20165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:34.491780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50ac3c-cff7-465e-a862-e0a41bc20165 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:34.492054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50ac3c-cff7-465e-a862-e0a41bc20165 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:34.492342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50ac3c-cff7-465e-a862-e0a41bc20165 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:34.558884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50ac3c-cff7-465e-a862-e0a41bc20165 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:34.560336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50ac3c-cff7-465e-a862-e0a41bc20165 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:34.654350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 587/1164] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:11:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:11:34.660215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceceb4d-8f35-49db-a9c0-c8cd3eca8c88 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:34.675217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:34.757137 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceceb4d-8f35-49db-a9c0-c8cd3eca8c88 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:34.790343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c6129-19f7-4c83-81da-5cf52cc76b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:34.790957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c6129-19f7-4c83-81da-5cf52cc76b85 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:34.791234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c6129-19f7-4c83-81da-5cf52cc76b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:34.791511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c6129-19f7-4c83-81da-5cf52cc76b85 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:34.797914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c6129-19f7-4c83-81da-5cf52cc76b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:34.799345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c6129-19f7-4c83-81da-5cf52cc76b85 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-a08c6129-19f7-4c83-81da-5cf52cc76b85 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:34.810388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:34.811702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:34.812895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 578/1165] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Sep 5 12:11:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:34.817357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50ac3c-cff7-465e-a862-e0a41bc20165 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:34.825401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:34.925772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50ac3c-cff7-465e-a862-e0a41bc20165 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:34.957725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fa53b-b313-4d8b-b765-330b1ba17f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:34.958840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fa53b-b313-4d8b-b765-330b1ba17f11 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:34.959116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fa53b-b313-4d8b-b765-330b1ba17f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:34.959386 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fa53b-b313-4d8b-b765-330b1ba17f11 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:34.960043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fa53b-b313-4d8b-b765-330b1ba17f11 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:34.960345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fa53b-b313-4d8b-b765-330b1ba17f11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:34.961683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fa53b-b313-4d8b-b765-330b1ba17f11 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:34.968545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 588/1166] 10.222.0.62 () {56 vars in 1115 bytes} [Mon Sep 5 12:11:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:36.849245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ecb73-3155-4e24-9d9d-e442cf838c2e None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:36.849818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ecb73-3155-4e24-9d9d-e442cf838c2e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:36.849818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ecb73-3155-4e24-9d9d-e442cf838c2e None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:36.851259 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 579/1167] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:36.856491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9693b2-1109-48b4-bdb9-0d865181737d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:36.856913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9693b2-1109-48b4-bdb9-0d865181737d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:36.857039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9693b2-1109-48b4-bdb9-0d865181737d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:36.874847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8f9693b2-1109-48b4-bdb9-0d865181737d None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:36.876053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8f9693b2-1109-48b4-bdb9-0d865181737d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:36.937923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:36.942170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 589/1168] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:37.211780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ecb73-3155-4e24-9d9d-e442cf838c2e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:37.285765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:37.310441 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ecb73-3155-4e24-9d9d-e442cf838c2e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:37.343693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53319abe-116a-4bdc-bcec-119b6fb0e350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:37.344279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53319abe-116a-4bdc-bcec-119b6fb0e350 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:37.344558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53319abe-116a-4bdc-bcec-119b6fb0e350 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:37.344837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53319abe-116a-4bdc-bcec-119b6fb0e350 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:37.345497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53319abe-116a-4bdc-bcec-119b6fb0e350 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:37.347121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53319abe-116a-4bdc-bcec-119b6fb0e350 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:37.551023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 580/1169] 10.222.0.62 () {60 vars in 1138 bytes} [Mon Sep 5 12:11:37 2022] POST /identity/v3/domains => generated 267 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:11:39.448239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ee19-cc56-468e-8de5-9a9c565a259e None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:39.448747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ee19-cc56-468e-8de5-9a9c565a259e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:39.448747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ee19-cc56-468e-8de5-9a9c565a259e None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:39.450072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 590/1170] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:39.455185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27c7cb-e7ff-4a53-a959-5c5ff82fb5df None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:39.455581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27c7cb-e7ff-4a53-a959-5c5ff82fb5df None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:39.455757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27c7cb-e7ff-4a53-a959-5c5ff82fb5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:39.474134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ea27c7cb-e7ff-4a53-a959-5c5ff82fb5df None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:39.475867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ea27c7cb-e7ff-4a53-a959-5c5ff82fb5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:39.539557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 581/1171] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:39.803396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ee19-cc56-468e-8de5-9a9c565a259e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:39.901877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ee19-cc56-468e-8de5-9a9c565a259e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:39.934248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b98136-4440-43d5-a5fc-843aadc2f097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:39.934859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b98136-4440-43d5-a5fc-843aadc2f097 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:39.935133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b98136-4440-43d5-a5fc-843aadc2f097 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:39.935400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b98136-4440-43d5-a5fc-843aadc2f097 None admin] PATH_INFO: `/v3/domains/3007ff59bfe44e88becbaf67b216d89b` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:39.945980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b98136-4440-43d5-a5fc-843aadc2f097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3007ff59bfe44e88becbaf67b216d89b)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:39.947399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b98136-4440-43d5-a5fc-843aadc2f097 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:39.948981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 591/1172] 10.222.0.62 () {56 vars in 1183 bytes} [Mon Sep 5 12:11:39 2022] GET /identity/v3/domains/3007ff59bfe44e88becbaf67b216d89b => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:39.954833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27c7cb-e7ff-4a53-a959-5c5ff82fb5df None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:40.015280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:40.050690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27c7cb-e7ff-4a53-a959-5c5ff82fb5df None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:40.084655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cda04f-5e3d-4319-b73f-48079135b915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:40.085192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cda04f-5e3d-4319-b73f-48079135b915 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:40.085470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cda04f-5e3d-4319-b73f-48079135b915 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:40.085741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cda04f-5e3d-4319-b73f-48079135b915 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:40.086374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cda04f-5e3d-4319-b73f-48079135b915 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:40.087837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cda04f-5e3d-4319-b73f-48079135b915 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:40.207710 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 582/1173] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:11:39 2022] POST /identity/v3/projects => generated 361 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:11:42.114454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db987f-0253-4723-848d-5c1fb694b1d4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:42.114951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db987f-0253-4723-848d-5c1fb694b1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:42.115046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db987f-0253-4723-848d-5c1fb694b1d4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:42.116392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 592/1174] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:42.121680 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761770-b2f3-4400-a306-35f785bd6ec6 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:42.122049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761770-b2f3-4400-a306-35f785bd6ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:42.122228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761770-b2f3-4400-a306-35f785bd6ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:42.140651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-3e761770-b2f3-4400-a306-35f785bd6ec6 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:42.141841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3e761770-b2f3-4400-a306-35f785bd6ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:42.207499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 583/1175] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:42.478043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db987f-0253-4723-848d-5c1fb694b1d4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:42.536426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:42.578342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db987f-0253-4723-848d-5c1fb694b1d4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:42.611427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e44df3-bd90-4a9a-b250-33995063f3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:42.612064 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e44df3-bd90-4a9a-b250-33995063f3de None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:42.612338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e44df3-bd90-4a9a-b250-33995063f3de None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:42.612605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e44df3-bd90-4a9a-b250-33995063f3de None admin] PATH_INFO: `/v3/domains/3007ff59bfe44e88becbaf67b216d89b` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:42.613697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e44df3-bd90-4a9a-b250-33995063f3de None admin] RBAC: Authorizing `identity:get_domain(domain_id=3007ff59bfe44e88becbaf67b216d89b)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:42.615757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e44df3-bd90-4a9a-b250-33995063f3de None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:42.617294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 593/1176] 10.222.0.62 () {56 vars in 1183 bytes} [Mon Sep 5 12:11:42 2022] GET /identity/v3/domains/3007ff59bfe44e88becbaf67b216d89b => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:42.622968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e761770-b2f3-4400-a306-35f785bd6ec6 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:42.667086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:42.717546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e761770-b2f3-4400-a306-35f785bd6ec6 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:42.751572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6969e-4615-4e40-9a4f-2bb242178c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:42.752099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6969e-4615-4e40-9a4f-2bb242178c7f None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:42.752373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6969e-4615-4e40-9a4f-2bb242178c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:42.752641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6969e-4615-4e40-9a4f-2bb242178c7f None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:42.753282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6969e-4615-4e40-9a4f-2bb242178c7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:42.754689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6969e-4615-4e40-9a4f-2bb242178c7f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:42.841608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 584/1177] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:11:42 2022] POST /identity/v3/users => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:11:44.745473 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8012ba-29f9-49e3-93b3-1289dfa897b9 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:44.746003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8012ba-29f9-49e3-93b3-1289dfa897b9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:44.746003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8012ba-29f9-49e3-93b3-1289dfa897b9 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:44.747314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 594/1178] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:44.752531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29b18b-b96e-4af8-845f-0832458f85d1 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:44.752949 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29b18b-b96e-4af8-845f-0832458f85d1 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:44.753062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29b18b-b96e-4af8-845f-0832458f85d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:44.770853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-be29b18b-b96e-4af8-845f-0832458f85d1 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:44.772550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-be29b18b-b96e-4af8-845f-0832458f85d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:44.837376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 585/1179] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:45.104226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8012ba-29f9-49e3-93b3-1289dfa897b9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:45.204940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8012ba-29f9-49e3-93b3-1289dfa897b9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:45.230275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:45.237636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c6f3d-bcc5-4a58-8d64-8bcc9734b425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:45.238245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c6f3d-bcc5-4a58-8d64-8bcc9734b425 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:45.238520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c6f3d-bcc5-4a58-8d64-8bcc9734b425 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:45.238787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c6f3d-bcc5-4a58-8d64-8bcc9734b425 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:45.249408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c6f3d-bcc5-4a58-8d64-8bcc9734b425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-9f4c6f3d-bcc5-4a58-8d64-8bcc9734b425 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:11:45.256553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:45.257787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:11:45.258831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 595/1180] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:11:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:45.261777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29b18b-b96e-4af8-845f-0832458f85d1 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:45.291524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:45.364263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29b18b-b96e-4af8-845f-0832458f85d1 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:45.397186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df534e1-bb65-4616-8893-c6d726fe7dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:45.397782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df534e1-bb65-4616-8893-c6d726fe7dab None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:45.398056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df534e1-bb65-4616-8893-c6d726fe7dab None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:45.398325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df534e1-bb65-4616-8893-c6d726fe7dab None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:45.398977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df534e1-bb65-4616-8893-c6d726fe7dab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:45.399274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df534e1-bb65-4616-8893-c6d726fe7dab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:45.400653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df534e1-bb65-4616-8893-c6d726fe7dab None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:45.408340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 586/1181] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:11:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:45.414163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c6f3d-bcc5-4a58-8d64-8bcc9734b425 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:45.513131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c6f3d-bcc5-4a58-8d64-8bcc9734b425 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:45.545324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54029b4-f851-4408-9e55-a334999ca025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:45.545939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54029b4-f851-4408-9e55-a334999ca025 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:45.546217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54029b4-f851-4408-9e55-a334999ca025 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:45.546489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54029b4-f851-4408-9e55-a334999ca025 None admin] PATH_INFO: `/v3/users/4191681bc79a4fbab90390c882c5b1f3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:45.559780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54029b4-f851-4408-9e55-a334999ca025 None admin] RBAC: Authorizing `identity:get_user(user_id=4191681bc79a4fbab90390c882c5b1f3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:45.561233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54029b4-f851-4408-9e55-a334999ca025 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:45.562801 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 596/1182] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:45 2022] GET /identity/v3/users/4191681bc79a4fbab90390c882c5b1f3 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:45.569367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df534e1-bb65-4616-8893-c6d726fe7dab None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:45.666189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:45.672690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df534e1-bb65-4616-8893-c6d726fe7dab None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:45.706099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e241-17e3-43cc-b216-17c248932d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:45.706695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e241-17e3-43cc-b216-17c248932d00 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:45.706978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e241-17e3-43cc-b216-17c248932d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:45.707254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e241-17e3-43cc-b216-17c248932d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:45.772108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e241-17e3-43cc-b216-17c248932d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:45.773550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e241-17e3-43cc-b216-17c248932d00 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:45.862166 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:45.869574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 587/1183] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:11:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:11:45.875552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54029b4-f851-4408-9e55-a334999ca025 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:45.921177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:45.972773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54029b4-f851-4408-9e55-a334999ca025 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:46.005799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9657b-b5d4-47ef-bc4a-7d6dcb386c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:46.006444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9657b-b5d4-47ef-bc4a-7d6dcb386c2c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:46.006722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9657b-b5d4-47ef-bc4a-7d6dcb386c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:46.006994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9657b-b5d4-47ef-bc4a-7d6dcb386c2c None admin] PATH_INFO: `/v3/projects/f1f04b3d6c564f38ae2ede09c710e722` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:46.008120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9657b-b5d4-47ef-bc4a-7d6dcb386c2c None admin] RBAC: Authorizing `identity:get_project(project_id=f1f04b3d6c564f38ae2ede09c710e722)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:46.009545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9657b-b5d4-47ef-bc4a-7d6dcb386c2c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:46.011138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 597/1184] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:45 2022] GET /identity/v3/projects/f1f04b3d6c564f38ae2ede09c710e722 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:46.017186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e241-17e3-43cc-b216-17c248932d00 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:46.113797 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e241-17e3-43cc-b216-17c248932d00 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:46.146316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d4deb-42cc-4bf4-90cc-d1418fc55a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:46.147426 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d4deb-42cc-4bf4-90cc-d1418fc55a5d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:46.147740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d4deb-42cc-4bf4-90cc-d1418fc55a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:46.148014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d4deb-42cc-4bf4-90cc-d1418fc55a5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:46.148722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d4deb-42cc-4bf4-90cc-d1418fc55a5d None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=f1f04b3d6c564f38ae2ede09c710e722, user.id=4191681bc79a4fbab90390c882c5b1f3) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:46.149026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d4deb-42cc-4bf4-90cc-d1418fc55a5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:46.150463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d4deb-42cc-4bf4-90cc-d1418fc55a5d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:46.155735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 588/1185] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:11:46 2022] GET /identity/v3/role_assignments?user.id=4191681bc79a4fbab90390c882c5b1f3&scope.project.id=f1f04b3d6c564f38ae2ede09c710e722&role.id=705907b9bdec47cebf88b868a6a56356 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:48.011119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c52ded-3ba2-4fe1-9791-ff954962387c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:48.011686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c52ded-3ba2-4fe1-9791-ff954962387c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:48.011686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c52ded-3ba2-4fe1-9791-ff954962387c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:48.013005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 598/1186] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:48.018155 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbce2a5-8fc6-49e2-9d21-1430f61067ac None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:48.018536 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbce2a5-8fc6-49e2-9d21-1430f61067ac None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:48.018692 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbce2a5-8fc6-49e2-9d21-1430f61067ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:48.037064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-5fbce2a5-8fc6-49e2-9d21-1430f61067ac None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:48.038271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5fbce2a5-8fc6-49e2-9d21-1430f61067ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:48.090283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:48.103618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 589/1187] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:48.371779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c52ded-3ba2-4fe1-9791-ff954962387c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:48.471330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c52ded-3ba2-4fe1-9791-ff954962387c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:48.485098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:48.504077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f07c9-bf7e-4ed6-8850-35ee933f3b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:48.504689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f07c9-bf7e-4ed6-8850-35ee933f3b1c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:48.504965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f07c9-bf7e-4ed6-8850-35ee933f3b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:48.505233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f07c9-bf7e-4ed6-8850-35ee933f3b1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:48.515769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f07c9-bf7e-4ed6-8850-35ee933f3b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-0e9f07c9-bf7e-4ed6-8850-35ee933f3b1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:11:48.520789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:48.522482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:11:48.523650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 599/1188] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:11:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:48.528489 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbce2a5-8fc6-49e2-9d21-1430f61067ac None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:48.624141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbce2a5-8fc6-49e2-9d21-1430f61067ac None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:48.656587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a0cb0-c8cd-46bc-9b62-b9189cf06bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:48.657177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0cb0-c8cd-46bc-9b62-b9189cf06bcd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:48.657455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0cb0-c8cd-46bc-9b62-b9189cf06bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:48.657725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0cb0-c8cd-46bc-9b62-b9189cf06bcd None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:48.658372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a0cb0-c8cd-46bc-9b62-b9189cf06bcd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:48.658672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a0cb0-c8cd-46bc-9b62-b9189cf06bcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:48.660224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a0cb0-c8cd-46bc-9b62-b9189cf06bcd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:48.668138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 590/1189] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:11:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:48.673712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f07c9-bf7e-4ed6-8850-35ee933f3b1c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:48.771281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f07c9-bf7e-4ed6-8850-35ee933f3b1c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:48.804115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea044-97ed-4e8a-bade-52538ca8b27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:48.804713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea044-97ed-4e8a-bade-52538ca8b27b None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:48.804992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea044-97ed-4e8a-bade-52538ca8b27b None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:48.805263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea044-97ed-4e8a-bade-52538ca8b27b None admin] PATH_INFO: `/v3/users/4191681bc79a4fbab90390c882c5b1f3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:48.806358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bea044-97ed-4e8a-bade-52538ca8b27b None admin] RBAC: Authorizing `identity:get_user(user_id=4191681bc79a4fbab90390c882c5b1f3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:48.807778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bea044-97ed-4e8a-bade-52538ca8b27b None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:48.809296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 600/1190] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:48 2022] GET /identity/v3/users/4191681bc79a4fbab90390c882c5b1f3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:48.814793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a0cb0-c8cd-46bc-9b62-b9189cf06bcd None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:48.863849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:48.912852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a0cb0-c8cd-46bc-9b62-b9189cf06bcd None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:48.945916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f447e1-c505-4756-9978-260c59879e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:48.946523 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f447e1-c505-4756-9978-260c59879e7f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:48.946800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f447e1-c505-4756-9978-260c59879e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:48.947073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f447e1-c505-4756-9978-260c59879e7f None admin] PATH_INFO: `/v3/projects/f1f04b3d6c564f38ae2ede09c710e722` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:48.948188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f447e1-c505-4756-9978-260c59879e7f None admin] RBAC: Authorizing `identity:get_project(project_id=f1f04b3d6c564f38ae2ede09c710e722)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:48.949612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f447e1-c505-4756-9978-260c59879e7f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:48.951209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 591/1191] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:48 2022] GET /identity/v3/projects/f1f04b3d6c564f38ae2ede09c710e722 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:48.956906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea044-97ed-4e8a-bade-52538ca8b27b None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:49.000246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:49.056381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea044-97ed-4e8a-bade-52538ca8b27b None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:49.090221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3c94c-ce1e-4c2f-8c3f-4f7d8f54cffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:49.090883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3c94c-ce1e-4c2f-8c3f-4f7d8f54cffc None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:49.091159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3c94c-ce1e-4c2f-8c3f-4f7d8f54cffc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:49.091455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3c94c-ce1e-4c2f-8c3f-4f7d8f54cffc None admin] PATH_INFO: `/v3/projects/f1f04b3d6c564f38ae2ede09c710e722/users/4191681bc79a4fbab90390c882c5b1f3/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:49.093427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3c94c-ce1e-4c2f-8c3f-4f7d8f54cffc None admin] RBAC: Authorizing `identity:create_grant(project_id=f1f04b3d6c564f38ae2ede09c710e722, user_id=4191681bc79a4fbab90390c882c5b1f3, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:49.094933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3c94c-ce1e-4c2f-8c3f-4f7d8f54cffc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:49.167859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 601/1192] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:11:48 2022] PUT /identity/v3/projects/f1f04b3d6c564f38ae2ede09c710e722/users/4191681bc79a4fbab90390c882c5b1f3/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:11:51.099087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8391b81-9ef2-4308-a8fd-e00b8b7a585a None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:51.099574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8391b81-9ef2-4308-a8fd-e00b8b7a585a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:51.099684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8391b81-9ef2-4308-a8fd-e00b8b7a585a None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:51.101018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 592/1193] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:11:51.106150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c7817-a7c6-4e02-bdf9-609a1f5be2e3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:51.106534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c7817-a7c6-4e02-bdf9-609a1f5be2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:51.106685 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c7817-a7c6-4e02-bdf9-609a1f5be2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:51.124765 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ac1c7817-a7c6-4e02-bdf9-609a1f5be2e3 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:11:51.125926 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ac1c7817-a7c6-4e02-bdf9-609a1f5be2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:11:51.191292 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 602/1194] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:11:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:11:51.462560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8391b81-9ef2-4308-a8fd-e00b8b7a585a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:51.567560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8391b81-9ef2-4308-a8fd-e00b8b7a585a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:51.585522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:51.602953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b1e5f-aa7f-4c92-91c6-bcf90ec3b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:51.603608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b1e5f-aa7f-4c92-91c6-bcf90ec3b34e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:51.603891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b1e5f-aa7f-4c92-91c6-bcf90ec3b34e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:51.604160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b1e5f-aa7f-4c92-91c6-bcf90ec3b34e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:51.614607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b1e5f-aa7f-4c92-91c6-bcf90ec3b34e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-a42b1e5f-aa7f-4c92-91c6-bcf90ec3b34e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:11:51.619389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:11:51.620528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:11:51.621370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 593/1195] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:11:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:11:51.626741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c7817-a7c6-4e02-bdf9-609a1f5be2e3 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:51.657327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:51.727011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c7817-a7c6-4e02-bdf9-609a1f5be2e3 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:51.759374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbc7ab-1374-48e3-a7e2-fe283c90eb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:51.760007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc7ab-1374-48e3-a7e2-fe283c90eb19 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:51.760288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc7ab-1374-48e3-a7e2-fe283c90eb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:51.760555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc7ab-1374-48e3-a7e2-fe283c90eb19 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:51.761211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbc7ab-1374-48e3-a7e2-fe283c90eb19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:51.761509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbc7ab-1374-48e3-a7e2-fe283c90eb19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:51.762857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbc7ab-1374-48e3-a7e2-fe283c90eb19 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:51.770626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 603/1196] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:11:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:11:51.777013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b1e5f-aa7f-4c92-91c6-bcf90ec3b34e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:51.875702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b1e5f-aa7f-4c92-91c6-bcf90ec3b34e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:51.909046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20122a53-01c2-4b84-a397-f5c91408b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:51.909650 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20122a53-01c2-4b84-a397-f5c91408b531 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:51.909927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20122a53-01c2-4b84-a397-f5c91408b531 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:51.910199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20122a53-01c2-4b84-a397-f5c91408b531 None admin] PATH_INFO: `/v3/users/4191681bc79a4fbab90390c882c5b1f3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:51.911298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20122a53-01c2-4b84-a397-f5c91408b531 None admin] RBAC: Authorizing `identity:get_user(user_id=4191681bc79a4fbab90390c882c5b1f3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:51.913333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20122a53-01c2-4b84-a397-f5c91408b531 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:51.914878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 594/1197] 10.222.0.62 () {56 vars in 1177 bytes} [Mon Sep 5 12:11:51 2022] GET /identity/v3/users/4191681bc79a4fbab90390c882c5b1f3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:51.920782 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbc7ab-1374-48e3-a7e2-fe283c90eb19 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:52.011727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:52.018062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbc7ab-1374-48e3-a7e2-fe283c90eb19 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:52.051066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28feb500-f54a-4bff-8850-520bd7e6df89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:52.051705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28feb500-f54a-4bff-8850-520bd7e6df89 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:52.052007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28feb500-f54a-4bff-8850-520bd7e6df89 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:52.052271 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28feb500-f54a-4bff-8850-520bd7e6df89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:52.117024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28feb500-f54a-4bff-8850-520bd7e6df89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:52.118438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28feb500-f54a-4bff-8850-520bd7e6df89 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:52.205121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:52.212671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 604/1198] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:11:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:11:52.218615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20122a53-01c2-4b84-a397-f5c91408b531 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:52.263432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:11:52.314087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20122a53-01c2-4b84-a397-f5c91408b531 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:52.346803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a222df8-1144-4dc8-ac8d-0f9f377e7317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:52.347406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a222df8-1144-4dc8-ac8d-0f9f377e7317 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:52.347710 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a222df8-1144-4dc8-ac8d-0f9f377e7317 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:52.348238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a222df8-1144-4dc8-ac8d-0f9f377e7317 None admin] PATH_INFO: `/v3/projects/f1f04b3d6c564f38ae2ede09c710e722` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:52.349330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a222df8-1144-4dc8-ac8d-0f9f377e7317 None admin] RBAC: Authorizing `identity:get_project(project_id=f1f04b3d6c564f38ae2ede09c710e722)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:52.350929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a222df8-1144-4dc8-ac8d-0f9f377e7317 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:52.352541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 595/1199] 10.222.0.62 () {56 vars in 1186 bytes} [Mon Sep 5 12:11:52 2022] GET /identity/v3/projects/f1f04b3d6c564f38ae2ede09c710e722 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:11:52.358665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28feb500-f54a-4bff-8850-520bd7e6df89 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:11:52.455724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28feb500-f54a-4bff-8850-520bd7e6df89 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:11:52.487893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e2797-3661-4465-a785-37f85fe5fcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:11:52.488482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e2797-3661-4465-a785-37f85fe5fcad None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:11:52.488755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e2797-3661-4465-a785-37f85fe5fcad None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:11:52.489027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e2797-3661-4465-a785-37f85fe5fcad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:11:52.489732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e2797-3661-4465-a785-37f85fe5fcad None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=f1f04b3d6c564f38ae2ede09c710e722, user.id=4191681bc79a4fbab90390c882c5b1f3) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:11:52.490030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e2797-3661-4465-a785-37f85fe5fcad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:11:52.491997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e2797-3661-4465-a785-37f85fe5fcad None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:11:52.497494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 605/1200] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:11:52 2022] GET /identity/v3/role_assignments?user.id=4191681bc79a4fbab90390c882c5b1f3&scope.project.id=f1f04b3d6c564f38ae2ede09c710e722&role.id=705907b9bdec47cebf88b868a6a56356 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:12:02.342889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=43 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:12:02.495016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=71 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:04.608491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35ab12-55f6-45c6-ab4c-c0ba11cdcba6 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:04.609070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35ab12-55f6-45c6-ab4c-c0ba11cdcba6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:04.609070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35ab12-55f6-45c6-ab4c-c0ba11cdcba6 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:04.610411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 596/1201] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:13:04.616073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e543450-dd41-4986-ac02-54d39c828f5e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:04.621462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e543450-dd41-4986-ac02-54d39c828f5e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:04.621462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e543450-dd41-4986-ac02-54d39c828f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:04.634367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3e543450-dd41-4986-ac02-54d39c828f5e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:04.635560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3e543450-dd41-4986-ac02-54d39c828f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:04.701074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 606/1202] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:04.966393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35ab12-55f6-45c6-ab4c-c0ba11cdcba6 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:05.070072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35ab12-55f6-45c6-ab4c-c0ba11cdcba6 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:05.105006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ab0a1-ad71-4c4a-bf12-8fe5ed4a54cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:05.105629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab0a1-ad71-4c4a-bf12-8fe5ed4a54cb None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:05.106163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab0a1-ad71-4c4a-bf12-8fe5ed4a54cb None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:05.106439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab0a1-ad71-4c4a-bf12-8fe5ed4a54cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:05.116507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ab0a1-ad71-4c4a-bf12-8fe5ed4a54cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:05.118012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ab0a1-ad71-4c4a-bf12-8fe5ed4a54cb None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-890ab0a1-ad71-4c4a-bf12-8fe5ed4a54cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:05.132846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:05.135407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:05.136662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:05.136662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:05.136662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:05.136662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 597/1203] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:05.140288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e543450-dd41-4986-ac02-54d39c828f5e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:05.238971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e543450-dd41-4986-ac02-54d39c828f5e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:05.266160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:05.272278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272e136-7c77-4359-a95a-be24d23c8d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:05.272878 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272e136-7c77-4359-a95a-be24d23c8d88 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:05.273156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272e136-7c77-4359-a95a-be24d23c8d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:05.273425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272e136-7c77-4359-a95a-be24d23c8d88 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:05.274061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272e136-7c77-4359-a95a-be24d23c8d88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:05.274360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272e136-7c77-4359-a95a-be24d23c8d88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:05.275749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272e136-7c77-4359-a95a-be24d23c8d88 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:05.287191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 607/1204] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:05.293174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ab0a1-ad71-4c4a-bf12-8fe5ed4a54cb None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:05.349589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:05.395545 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ab0a1-ad71-4c4a-bf12-8fe5ed4a54cb None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:05.426769 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b6379-409d-4f19-8d96-08b9e7dc9eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:05.427358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b6379-409d-4f19-8d96-08b9e7dc9eb6 None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:05.427662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b6379-409d-4f19-8d96-08b9e7dc9eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:05.427978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b6379-409d-4f19-8d96-08b9e7dc9eb6 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:05.428622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b6379-409d-4f19-8d96-08b9e7dc9eb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:05.430042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b6379-409d-4f19-8d96-08b9e7dc9eb6 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:05.534439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 598/1205] 10.222.0.62 () {60 vars in 1132 bytes} [Mon Sep 5 12:13:05 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:13:07.475837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949db8f-8d35-4393-b9e6-9c9354e446f1 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:07.476369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949db8f-8d35-4393-b9e6-9c9354e446f1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:07.476369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949db8f-8d35-4393-b9e6-9c9354e446f1 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:07.477735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 608/1206] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:07.483268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416378e7-c333-408a-9ce9-5e60a88924ff None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:07.483689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416378e7-c333-408a-9ce9-5e60a88924ff None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:07.483839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416378e7-c333-408a-9ce9-5e60a88924ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:07.502250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-416378e7-c333-408a-9ce9-5e60a88924ff None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:07.503418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-416378e7-c333-408a-9ce9-5e60a88924ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:07.569763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 599/1207] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:07.841283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949db8f-8d35-4393-b9e6-9c9354e446f1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:07.945730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949db8f-8d35-4393-b9e6-9c9354e446f1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:07.979918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487935fc-a811-4c35-9512-cb9535f4e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:07.980543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487935fc-a811-4c35-9512-cb9535f4e464 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:07.980822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487935fc-a811-4c35-9512-cb9535f4e464 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:07.981361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487935fc-a811-4c35-9512-cb9535f4e464 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:07.992545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487935fc-a811-4c35-9512-cb9535f4e464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-487935fc-a811-4c35-9512-cb9535f4e464 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:13:07.997298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:07.998428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:07.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 609/1208] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:13:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:08.005961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416378e7-c333-408a-9ce9-5e60a88924ff None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:08.045827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:08.104153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416378e7-c333-408a-9ce9-5e60a88924ff None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:08.138363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4bfa8-92ad-41ce-a00b-7ef8651f998d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:08.138977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4bfa8-92ad-41ce-a00b-7ef8651f998d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:08.139237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4bfa8-92ad-41ce-a00b-7ef8651f998d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:08.139506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4bfa8-92ad-41ce-a00b-7ef8651f998d None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:08.140189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4bfa8-92ad-41ce-a00b-7ef8651f998d None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:08.140486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4bfa8-92ad-41ce-a00b-7ef8651f998d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:08.141849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4bfa8-92ad-41ce-a00b-7ef8651f998d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:08.149742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 600/1209] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:13:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:08.155516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487935fc-a811-4c35-9512-cb9535f4e464 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:08.205192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:08.254757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487935fc-a811-4c35-9512-cb9535f4e464 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:08.286268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd1d29-6f85-435d-9f0b-eb7e3ec56343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:08.286876 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd1d29-6f85-435d-9f0b-eb7e3ec56343 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:08.287150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd1d29-6f85-435d-9f0b-eb7e3ec56343 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:08.287416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd1d29-6f85-435d-9f0b-eb7e3ec56343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:08.297382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd1d29-6f85-435d-9f0b-eb7e3ec56343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:08.298801 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd1d29-6f85-435d-9f0b-eb7e3ec56343 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-d8cd1d29-6f85-435d-9f0b-eb7e3ec56343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:08.313113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:08.314270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:08.315689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:08.315689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:08.315689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:08.315689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 610/1210] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:08.320939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4bfa8-92ad-41ce-a00b-7ef8651f998d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:08.426467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:08.429420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4bfa8-92ad-41ce-a00b-7ef8651f998d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:08.461762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9c321-a0b3-411e-9c07-691ff3c5396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:08.462361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9c321-a0b3-411e-9c07-691ff3c5396e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:08.462639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9c321-a0b3-411e-9c07-691ff3c5396e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:08.462911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9c321-a0b3-411e-9c07-691ff3c5396e None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:08.463565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9c321-a0b3-411e-9c07-691ff3c5396e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:08.463869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9c321-a0b3-411e-9c07-691ff3c5396e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:08.465233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9c321-a0b3-411e-9c07-691ff3c5396e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:08.476401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 601/1211] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:08.482462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd1d29-6f85-435d-9f0b-eb7e3ec56343 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:08.580483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd1d29-6f85-435d-9f0b-eb7e3ec56343 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:08.581996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:08.614373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c8d31-8713-4a7e-956f-dc8c875b7bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:08.614992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c8d31-8713-4a7e-956f-dc8c875b7bca None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:08.615273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c8d31-8713-4a7e-956f-dc8c875b7bca None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:08.615569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c8d31-8713-4a7e-956f-dc8c875b7bca None admin] PATH_INFO: `/v3/users/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:08.623121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c8d31-8713-4a7e-956f-dc8c875b7bca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:08.624613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c8d31-8713-4a7e-956f-dc8c875b7bca None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-187c8d31-8713-4a7e-956f-dc8c875b7bca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:13:08.635741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:08.636908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:08.638061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 611/1212] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:08.642712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9c321-a0b3-411e-9c07-691ff3c5396e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:08.742808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9c321-a0b3-411e-9c07-691ff3c5396e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:08.775032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686e290-6f34-4f2d-8fbe-73d47d0d7d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:08.775634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686e290-6f34-4f2d-8fbe-73d47d0d7d08 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:08.775910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686e290-6f34-4f2d-8fbe-73d47d0d7d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:08.776177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686e290-6f34-4f2d-8fbe-73d47d0d7d08 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:08.776883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686e290-6f34-4f2d-8fbe-73d47d0d7d08 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:08.777180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686e290-6f34-4f2d-8fbe-73d47d0d7d08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:08.778598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686e290-6f34-4f2d-8fbe-73d47d0d7d08 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:08.795514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 602/1213] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Sep 5 12:13:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:08.801257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c8d31-8713-4a7e-956f-dc8c875b7bca None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:08.902091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c8d31-8713-4a7e-956f-dc8c875b7bca None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:08.935037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a5b8-2b96-4cd5-9e35-1629dc6017fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:08.935659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a5b8-2b96-4cd5-9e35-1629dc6017fc None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:08.935934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a5b8-2b96-4cd5-9e35-1629dc6017fc None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:08.936202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a5b8-2b96-4cd5-9e35-1629dc6017fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:08.965839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:09.002470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247a5b8-2b96-4cd5-9e35-1629dc6017fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:09.004448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247a5b8-2b96-4cd5-9e35-1629dc6017fc None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:09.101118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 612/1214] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:13:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:13:09.107167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686e290-6f34-4f2d-8fbe-73d47d0d7d08 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:09.126281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:09.205704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686e290-6f34-4f2d-8fbe-73d47d0d7d08 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:09.241470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b114e-a8df-4ccd-8b87-8c215137d1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:09.242096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b114e-a8df-4ccd-8b87-8c215137d1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:09.242372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b114e-a8df-4ccd-8b87-8c215137d1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:09.242643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b114e-a8df-4ccd-8b87-8c215137d1a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:09.252349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b114e-a8df-4ccd-8b87-8c215137d1a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:09.262293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b114e-a8df-4ccd-8b87-8c215137d1a7 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-567b114e-a8df-4ccd-8b87-8c215137d1a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:09.268384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:09.270072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:09.271228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:09.271228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:09.271228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:09.271228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 603/1215] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:09.276035 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a5b8-2b96-4cd5-9e35-1629dc6017fc None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:09.335522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:09.374141 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a5b8-2b96-4cd5-9e35-1629dc6017fc None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:09.408203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345df56e-1e25-4ea3-b846-f3a7b4991a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:09.408803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345df56e-1e25-4ea3-b846-f3a7b4991a91 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:09.409085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345df56e-1e25-4ea3-b846-f3a7b4991a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:09.409353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345df56e-1e25-4ea3-b846-f3a7b4991a91 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:09.410013 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345df56e-1e25-4ea3-b846-f3a7b4991a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:09.410314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345df56e-1e25-4ea3-b846-f3a7b4991a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:09.411693 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345df56e-1e25-4ea3-b846-f3a7b4991a91 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:09.422876 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 613/1216] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:09.429217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b114e-a8df-4ccd-8b87-8c215137d1a7 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:09.499005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:09.531004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b114e-a8df-4ccd-8b87-8c215137d1a7 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:09.565848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f993-41cf-4f1d-9633-06950543134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:09.567111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f993-41cf-4f1d-9633-06950543134f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:09.567425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f993-41cf-4f1d-9633-06950543134f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:09.567761 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f993-41cf-4f1d-9633-06950543134f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:09.574412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29f993-41cf-4f1d-9633-06950543134f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:09.575930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29f993-41cf-4f1d-9633-06950543134f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-9e29f993-41cf-4f1d-9633-06950543134f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:13:09.586956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:09.588065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:09.589350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:13:09.589350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:13:09.589350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:13:09.589350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:13:09.589350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:13:09.589350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:13:09.589350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:09.589350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:09.589350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 604/1217] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:13:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:09.594034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345df56e-1e25-4ea3-b846-f3a7b4991a91 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:09.693595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345df56e-1e25-4ea3-b846-f3a7b4991a91 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:09.715503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:09.726769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77858aeb-2a21-4219-9ff6-9f2dffa945ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:09.727374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77858aeb-2a21-4219-9ff6-9f2dffa945ba None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:09.727686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77858aeb-2a21-4219-9ff6-9f2dffa945ba None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:09.727958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77858aeb-2a21-4219-9ff6-9f2dffa945ba None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:09.728643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77858aeb-2a21-4219-9ff6-9f2dffa945ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:09.728943 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77858aeb-2a21-4219-9ff6-9f2dffa945ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:09.730349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77858aeb-2a21-4219-9ff6-9f2dffa945ba None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:09.742702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 614/1218] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:13:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:09.748873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f993-41cf-4f1d-9633-06950543134f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:09.850616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f993-41cf-4f1d-9633-06950543134f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:09.879418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:09.885052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9abb6-f7a3-4bcd-8b29-a68f9125ef5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:09.885731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9abb6-f7a3-4bcd-8b29-a68f9125ef5e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:09.886049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9abb6-f7a3-4bcd-8b29-a68f9125ef5e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:09.886358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9abb6-f7a3-4bcd-8b29-a68f9125ef5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:09.887110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9abb6-f7a3-4bcd-8b29-a68f9125ef5e None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=90d65372febb414d82843f9dfa56f738) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:09.887454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9abb6-f7a3-4bcd-8b29-a68f9125ef5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:09.888957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9abb6-f7a3-4bcd-8b29-a68f9125ef5e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:09.894289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 605/1219] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:13:09 2022] GET /identity/v3/role_assignments?user.id=90d65372febb414d82843f9dfa56f738&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:11.768818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bf68b-e7af-41f6-b81c-b71a46f08a53 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:11.769348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bf68b-e7af-41f6-b81c-b71a46f08a53 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:11.769348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bf68b-e7af-41f6-b81c-b71a46f08a53 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:11.770725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 615/1220] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:11.776134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc83f2-9d55-4ddb-b3da-ad9488218b17 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:11.776554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc83f2-9d55-4ddb-b3da-ad9488218b17 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:11.776884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc83f2-9d55-4ddb-b3da-ad9488218b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:11.795390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-b6cc83f2-9d55-4ddb-b3da-ad9488218b17 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:11.797958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b6cc83f2-9d55-4ddb-b3da-ad9488218b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:11.865269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 606/1221] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:12.139310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bf68b-e7af-41f6-b81c-b71a46f08a53 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:12.239841 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bf68b-e7af-41f6-b81c-b71a46f08a53 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:12.272055 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f7c27-155a-443f-876d-334866759ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:12.272672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f7c27-155a-443f-876d-334866759ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:12.272946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f7c27-155a-443f-876d-334866759ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:12.273214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f7c27-155a-443f-876d-334866759ed0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:12.282857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f7c27-155a-443f-876d-334866759ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-179f7c27-155a-443f-876d-334866759ed0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:13:12.288096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:12.289212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:12.290279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 616/1222] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:13:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:12.295232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc83f2-9d55-4ddb-b3da-ad9488218b17 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:12.396229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc83f2-9d55-4ddb-b3da-ad9488218b17 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:12.420017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:12.431522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38658db5-d884-4e0a-8c6f-a7e97c9b0fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:12.432186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38658db5-d884-4e0a-8c6f-a7e97c9b0fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:12.432501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38658db5-d884-4e0a-8c6f-a7e97c9b0fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:12.432807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38658db5-d884-4e0a-8c6f-a7e97c9b0fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:12.433495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38658db5-d884-4e0a-8c6f-a7e97c9b0fb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:12.433835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38658db5-d884-4e0a-8c6f-a7e97c9b0fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:12.435234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38658db5-d884-4e0a-8c6f-a7e97c9b0fb0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:12.443499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 607/1223] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:13:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:12.449247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f7c27-155a-443f-876d-334866759ed0 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:12.493388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:12.548501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f7c27-155a-443f-876d-334866759ed0 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:12.580683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894ea49-0bc0-49f7-b1ec-28066ed07d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:12.583583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894ea49-0bc0-49f7-b1ec-28066ed07d16 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:12.583583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894ea49-0bc0-49f7-b1ec-28066ed07d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:12.583583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894ea49-0bc0-49f7-b1ec-28066ed07d16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:12.592003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894ea49-0bc0-49f7-b1ec-28066ed07d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:12.593411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894ea49-0bc0-49f7-b1ec-28066ed07d16 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-6894ea49-0bc0-49f7-b1ec-28066ed07d16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:12.608134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:12.613251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:12.614391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:12.614391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:12.614391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:12.614391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 617/1224] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:12.614997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38658db5-d884-4e0a-8c6f-a7e97c9b0fb0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:12.719673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38658db5-d884-4e0a-8c6f-a7e97c9b0fb0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:12.754558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7941a-3b8e-4c9d-bc92-d9f7bc119a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:12.755191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7941a-3b8e-4c9d-bc92-d9f7bc119a05 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:12.755508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7941a-3b8e-4c9d-bc92-d9f7bc119a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:12.755883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7941a-3b8e-4c9d-bc92-d9f7bc119a05 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:12.756554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7941a-3b8e-4c9d-bc92-d9f7bc119a05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:12.756895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7941a-3b8e-4c9d-bc92-d9f7bc119a05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:12.758301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7941a-3b8e-4c9d-bc92-d9f7bc119a05 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:12.770406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 608/1225] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:13:12.776403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894ea49-0bc0-49f7-b1ec-28066ed07d16 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:12.869587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:12.876103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894ea49-0bc0-49f7-b1ec-28066ed07d16 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:12.909389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcfb0b-897a-401f-a3b9-b4ce6ff9f4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:12.910008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcfb0b-897a-401f-a3b9-b4ce6ff9f4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:12.910284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcfb0b-897a-401f-a3b9-b4ce6ff9f4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:12.910554 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcfb0b-897a-401f-a3b9-b4ce6ff9f4d4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:12.917810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcfb0b-897a-401f-a3b9-b4ce6ff9f4d4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:12.919221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcfb0b-897a-401f-a3b9-b4ce6ff9f4d4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-cbfcfb0b-897a-401f-a3b9-b4ce6ff9f4d4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:13:12.930147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:12.931267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:12.932471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 618/1226] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:12.937347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7941a-3b8e-4c9d-bc92-d9f7bc119a05 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:12.985737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:13.037035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7941a-3b8e-4c9d-bc92-d9f7bc119a05 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:13.071746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad599ba-af0f-452b-b7f3-34d5363c5704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:13.072355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad599ba-af0f-452b-b7f3-34d5363c5704 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:13.072664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad599ba-af0f-452b-b7f3-34d5363c5704 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:13.072939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad599ba-af0f-452b-b7f3-34d5363c5704 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:13.073685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad599ba-af0f-452b-b7f3-34d5363c5704 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:13.073987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad599ba-af0f-452b-b7f3-34d5363c5704 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:13.075411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad599ba-af0f-452b-b7f3-34d5363c5704 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:13.092775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 609/1227] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Sep 5 12:13:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:13.098631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcfb0b-897a-401f-a3b9-b4ce6ff9f4d4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:13.197388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcfb0b-897a-401f-a3b9-b4ce6ff9f4d4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:13.229716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613569b4-d74b-4ad7-8566-a035a9a84604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:13.230339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613569b4-d74b-4ad7-8566-a035a9a84604 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:13.230615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613569b4-d74b-4ad7-8566-a035a9a84604 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:13.230888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613569b4-d74b-4ad7-8566-a035a9a84604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:13.241918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613569b4-d74b-4ad7-8566-a035a9a84604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:13.243290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613569b4-d74b-4ad7-8566-a035a9a84604 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-613569b4-d74b-4ad7-8566-a035a9a84604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:13.257639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:13.259086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:13.260482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:13.260482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:13.260482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:13.260482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 619/1228] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:13.264596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad599ba-af0f-452b-b7f3-34d5363c5704 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:13.363176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad599ba-af0f-452b-b7f3-34d5363c5704 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:13.365965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:13.397553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb497f-82c8-4d81-9ebc-0f0254c7b297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:13.398211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb497f-82c8-4d81-9ebc-0f0254c7b297 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:13.398688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb497f-82c8-4d81-9ebc-0f0254c7b297 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:13.399194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb497f-82c8-4d81-9ebc-0f0254c7b297 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:13.400169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb497f-82c8-4d81-9ebc-0f0254c7b297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:13.400483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb497f-82c8-4d81-9ebc-0f0254c7b297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:13.401863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb497f-82c8-4d81-9ebc-0f0254c7b297 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:13.413032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 610/1229] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:13.418833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613569b4-d74b-4ad7-8566-a035a9a84604 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:13.428736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:13.517839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613569b4-d74b-4ad7-8566-a035a9a84604 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:13.550759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3e1ca-cd8a-4547-8906-4ddd2078b0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:13.552161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e1ca-cd8a-4547-8906-4ddd2078b0fb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:13.552447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e1ca-cd8a-4547-8906-4ddd2078b0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:13.552723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e1ca-cd8a-4547-8906-4ddd2078b0fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:13.559438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3e1ca-cd8a-4547-8906-4ddd2078b0fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:13.560930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3e1ca-cd8a-4547-8906-4ddd2078b0fb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-00a3e1ca-cd8a-4547-8906-4ddd2078b0fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:13:13.572134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:13.573172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:13.574071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:13:13.574071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:13:13.574071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:13:13.574071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:13:13.574071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:13:13.574071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:13:13.574071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:13.574071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:13.574071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 620/1230] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:13:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:13.579100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb497f-82c8-4d81-9ebc-0f0254c7b297 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:13.677970 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb497f-82c8-4d81-9ebc-0f0254c7b297 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:13.711674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24b24-6662-4082-8127-e02d212f71cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:13.712271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24b24-6662-4082-8127-e02d212f71cd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:13.712549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24b24-6662-4082-8127-e02d212f71cd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:13.712817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24b24-6662-4082-8127-e02d212f71cd None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:13.713499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24b24-6662-4082-8127-e02d212f71cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:13.713800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24b24-6662-4082-8127-e02d212f71cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:13.715223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24b24-6662-4082-8127-e02d212f71cd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:13.727301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 611/1231] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:13:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:13.733865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3e1ca-cd8a-4547-8906-4ddd2078b0fb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:13.800478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:13.835499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3e1ca-cd8a-4547-8906-4ddd2078b0fb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:13.869085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869f51a-1f1b-4e15-8392-f491510d732c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:13.869735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869f51a-1f1b-4e15-8392-f491510d732c None admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:13.870012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869f51a-1f1b-4e15-8392-f491510d732c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:13.870286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869f51a-1f1b-4e15-8392-f491510d732c None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/90d65372febb414d82843f9dfa56f738/roles/dd50550e774349d6af84f593eeeec10d` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:13.884557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869f51a-1f1b-4e15-8392-f491510d732c None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=90d65372febb414d82843f9dfa56f738, role_id=dd50550e774349d6af84f593eeeec10d)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:13.885982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869f51a-1f1b-4e15-8392-f491510d732c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:13.955018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 621/1232] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:13:13 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/90d65372febb414d82843f9dfa56f738/roles/dd50550e774349d6af84f593eeeec10d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:13:15.855117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ec05c-883c-47c0-b973-39c0c2d93c98 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:15.855712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ec05c-883c-47c0-b973-39c0c2d93c98 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:15.855712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ec05c-883c-47c0-b973-39c0c2d93c98 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:15.857501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 612/1233] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:15.862562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f07001-62d0-481b-a73a-b005b47e2120 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:15.863018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f07001-62d0-481b-a73a-b005b47e2120 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:15.863126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f07001-62d0-481b-a73a-b005b47e2120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:15.881511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a0f07001-62d0-481b-a73a-b005b47e2120 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:15.882689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a0f07001-62d0-481b-a73a-b005b47e2120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:15.947484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 622/1234] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:16.223454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ec05c-883c-47c0-b973-39c0c2d93c98 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:16.248767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:16.320867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ec05c-883c-47c0-b973-39c0c2d93c98 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:16.355597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555ea4e-29a5-4583-b19f-c2e85c1c653a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:16.356395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555ea4e-29a5-4583-b19f-c2e85c1c653a None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:16.357540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555ea4e-29a5-4583-b19f-c2e85c1c653a None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:16.357859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555ea4e-29a5-4583-b19f-c2e85c1c653a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:16.368241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555ea4e-29a5-4583-b19f-c2e85c1c653a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-e555ea4e-29a5-4583-b19f-c2e85c1c653a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:13:16.373042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:16.374101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:16.374965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 613/1235] 10.222.0.62 () {56 vars in 1102 bytes} [Mon Sep 5 12:13:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:16.380099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f07001-62d0-481b-a73a-b005b47e2120 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:16.425581 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:16.475665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f07001-62d0-481b-a73a-b005b47e2120 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:16.507762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67e8c4-ff51-4776-b5f5-15ecba3953e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:16.508354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67e8c4-ff51-4776-b5f5-15ecba3953e9 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:16.508628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67e8c4-ff51-4776-b5f5-15ecba3953e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:16.508896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67e8c4-ff51-4776-b5f5-15ecba3953e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:16.509543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67e8c4-ff51-4776-b5f5-15ecba3953e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:16.509839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67e8c4-ff51-4776-b5f5-15ecba3953e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:16.511179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67e8c4-ff51-4776-b5f5-15ecba3953e9 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:16.519234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 623/1236] 10.222.0.62 () {56 vars in 1103 bytes} [Mon Sep 5 12:13:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:16.525783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555ea4e-29a5-4583-b19f-c2e85c1c653a None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:16.609507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:16.624545 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555ea4e-29a5-4583-b19f-c2e85c1c653a None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:16.657760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97ac45-7d26-4354-9701-485bfe9ba5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:16.661562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97ac45-7d26-4354-9701-485bfe9ba5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:16.661562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97ac45-7d26-4354-9701-485bfe9ba5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:16.661562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97ac45-7d26-4354-9701-485bfe9ba5d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:16.668556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97ac45-7d26-4354-9701-485bfe9ba5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:16.669992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97ac45-7d26-4354-9701-485bfe9ba5d2 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-6b97ac45-7d26-4354-9701-485bfe9ba5d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:16.684808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:16.685921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:16.687022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:16.687022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:16.687022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:16.687022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 614/1237] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:16.691995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67e8c4-ff51-4776-b5f5-15ecba3953e9 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:16.790288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67e8c4-ff51-4776-b5f5-15ecba3953e9 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:16.799208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:16.823790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a24711-3946-4bbc-8076-d3fec7eec293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:16.824388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a24711-3946-4bbc-8076-d3fec7eec293 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:16.824666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a24711-3946-4bbc-8076-d3fec7eec293 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:16.824944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a24711-3946-4bbc-8076-d3fec7eec293 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:16.825611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a24711-3946-4bbc-8076-d3fec7eec293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:16.825910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a24711-3946-4bbc-8076-d3fec7eec293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:16.827255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a24711-3946-4bbc-8076-d3fec7eec293 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:16.839386 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 624/1238] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:16.845195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97ac45-7d26-4354-9701-485bfe9ba5d2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:16.945196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97ac45-7d26-4354-9701-485bfe9ba5d2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:16.977664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15614ba4-fe09-4f6e-9147-6e0806f2fc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:16.983140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15614ba4-fe09-4f6e-9147-6e0806f2fc9d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:16.983140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15614ba4-fe09-4f6e-9147-6e0806f2fc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:16.983140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15614ba4-fe09-4f6e-9147-6e0806f2fc9d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:16.983140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:16.986502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15614ba4-fe09-4f6e-9147-6e0806f2fc9d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:16.988029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15614ba4-fe09-4f6e-9147-6e0806f2fc9d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-15614ba4-fe09-4f6e-9147-6e0806f2fc9d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:13:16.999244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:17.000380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:17.001779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 615/1239] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:17.006725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a24711-3946-4bbc-8076-d3fec7eec293 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:17.104049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a24711-3946-4bbc-8076-d3fec7eec293 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:17.138385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7592ef0-7c99-41fc-b8db-0fcc28651a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:17.138988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7592ef0-7c99-41fc-b8db-0fcc28651a21 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:17.139267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7592ef0-7c99-41fc-b8db-0fcc28651a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:17.139557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7592ef0-7c99-41fc-b8db-0fcc28651a21 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:17.140313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7592ef0-7c99-41fc-b8db-0fcc28651a21 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:17.140614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7592ef0-7c99-41fc-b8db-0fcc28651a21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:17.142033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7592ef0-7c99-41fc-b8db-0fcc28651a21 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:17.159643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 625/1240] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Sep 5 12:13:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:17.165602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15614ba4-fe09-4f6e-9147-6e0806f2fc9d None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:17.265204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15614ba4-fe09-4f6e-9147-6e0806f2fc9d None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:17.297751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b552f61-58af-41ee-8423-a5599ad2b7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:17.298340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b552f61-58af-41ee-8423-a5599ad2b7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:17.298619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b552f61-58af-41ee-8423-a5599ad2b7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:17.298884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b552f61-58af-41ee-8423-a5599ad2b7f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:17.358254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:17.365112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b552f61-58af-41ee-8423-a5599ad2b7f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:17.366550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b552f61-58af-41ee-8423-a5599ad2b7f5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:17.462324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 616/1241] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:13:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:13:17.468221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7592ef0-7c99-41fc-b8db-0fcc28651a21 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:17.496555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:17.567263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7592ef0-7c99-41fc-b8db-0fcc28651a21 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:17.603260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4e0b7-155b-4ba6-bb12-86b56b45ec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:17.603920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4e0b7-155b-4ba6-bb12-86b56b45ec8d None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:17.604192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4e0b7-155b-4ba6-bb12-86b56b45ec8d None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:17.604459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4e0b7-155b-4ba6-bb12-86b56b45ec8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:17.615150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4e0b7-155b-4ba6-bb12-86b56b45ec8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:17.616534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4e0b7-155b-4ba6-bb12-86b56b45ec8d None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-72f4e0b7-155b-4ba6-bb12-86b56b45ec8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:17.630759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:17.631939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:17.633237 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:17.633237 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:17.633237 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:17.633237 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 626/1242] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:17.637553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b552f61-58af-41ee-8423-a5599ad2b7f5 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:17.731558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:17.737488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b552f61-58af-41ee-8423-a5599ad2b7f5 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:17.773429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344f8c-bda0-4aa8-af9a-692614495bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:17.774087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344f8c-bda0-4aa8-af9a-692614495bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:17.774367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344f8c-bda0-4aa8-af9a-692614495bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:17.774696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344f8c-bda0-4aa8-af9a-692614495bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:17.775332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b344f8c-bda0-4aa8-af9a-692614495bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:17.775712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b344f8c-bda0-4aa8-af9a-692614495bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:17.777239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b344f8c-bda0-4aa8-af9a-692614495bb4 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:17.789736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 617/1243] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:17.795823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4e0b7-155b-4ba6-bb12-86b56b45ec8d None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:17.873799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:17.894427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4e0b7-155b-4ba6-bb12-86b56b45ec8d None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:17.927970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fedb3-2001-46a0-8e9f-134d1722e27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:17.928663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fedb3-2001-46a0-8e9f-134d1722e27e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:17.928941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fedb3-2001-46a0-8e9f-134d1722e27e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:17.929208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fedb3-2001-46a0-8e9f-134d1722e27e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:17.935575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fedb3-2001-46a0-8e9f-134d1722e27e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:17.936998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fedb3-2001-46a0-8e9f-134d1722e27e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f95fedb3-2001-46a0-8e9f-134d1722e27e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:13:17.948142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:17.953999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:17.955243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:13:17.955243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:13:17.955243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:13:17.955243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:13:17.955243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:13:17.955243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:13:17.955243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:17.955243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:17.955243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 627/1244] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:17.955912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344f8c-bda0-4aa8-af9a-692614495bb4 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:18.056281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344f8c-bda0-4aa8-af9a-692614495bb4 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:18.090706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa11697-8df3-4f42-860a-35630ea5a124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:18.091313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa11697-8df3-4f42-860a-35630ea5a124 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:18.091621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa11697-8df3-4f42-860a-35630ea5a124 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:18.091893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa11697-8df3-4f42-860a-35630ea5a124 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:18.092598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa11697-8df3-4f42-860a-35630ea5a124 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:18.092885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa11697-8df3-4f42-860a-35630ea5a124 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:18.094347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa11697-8df3-4f42-860a-35630ea5a124 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:18.106199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 618/1245] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:13:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:18.112650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fedb3-2001-46a0-8e9f-134d1722e27e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:18.212914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fedb3-2001-46a0-8e9f-134d1722e27e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:18.246578 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747fcda-2882-4462-afd3-dc81358da7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:18.247178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747fcda-2882-4462-afd3-dc81358da7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:18.247459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747fcda-2882-4462-afd3-dc81358da7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:18.247761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747fcda-2882-4462-afd3-dc81358da7c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:18.248479 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747fcda-2882-4462-afd3-dc81358da7c6 None admin] RBAC: Adding query filter params (role.id=dd50550e774349d6af84f593eeeec10d, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=90d65372febb414d82843f9dfa56f738) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:18.248784 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747fcda-2882-4462-afd3-dc81358da7c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:18.250205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747fcda-2882-4462-afd3-dc81358da7c6 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:18.256380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 628/1246] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:13:18 2022] GET /identity/v3/role_assignments?user.id=90d65372febb414d82843f9dfa56f738&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=dd50550e774349d6af84f593eeeec10d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:20.190365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eb7f1-1817-4c93-8116-5c8ba6271de4 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:20.190913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eb7f1-1817-4c93-8116-5c8ba6271de4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:20.190913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eb7f1-1817-4c93-8116-5c8ba6271de4 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:20.192280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 619/1247] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:20.197509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2718c-f292-47ca-a896-8c8017b051af None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:20.197884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2718c-f292-47ca-a896-8c8017b051af None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:20.198046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2718c-f292-47ca-a896-8c8017b051af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:20.204507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:20.217079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-76f2718c-f292-47ca-a896-8c8017b051af None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:20.218258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-76f2718c-f292-47ca-a896-8c8017b051af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:20.282091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 629/1248] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:20.551287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eb7f1-1817-4c93-8116-5c8ba6271de4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:20.569664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:20.653447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eb7f1-1817-4c93-8116-5c8ba6271de4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:20.686934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd1e85-3f98-4f34-afcf-ca522c8afee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:20.687577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd1e85-3f98-4f34-afcf-ca522c8afee4 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:20.687862 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd1e85-3f98-4f34-afcf-ca522c8afee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:20.688136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd1e85-3f98-4f34-afcf-ca522c8afee4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:20.698833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd1e85-3f98-4f34-afcf-ca522c8afee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-eddd1e85-3f98-4f34-afcf-ca522c8afee4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:13:20.703660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:20.704608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:20.705519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 620/1249] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:13:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:20.710640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2718c-f292-47ca-a896-8c8017b051af None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:20.806901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2718c-f292-47ca-a896-8c8017b051af None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:20.836261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:20.841128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b667ab8-ff85-4c16-be6e-5639c6233bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:20.841726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b667ab8-ff85-4c16-be6e-5639c6233bdb None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:20.841998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b667ab8-ff85-4c16-be6e-5639c6233bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:20.842265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b667ab8-ff85-4c16-be6e-5639c6233bdb None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:20.842907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b667ab8-ff85-4c16-be6e-5639c6233bdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:20.843199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b667ab8-ff85-4c16-be6e-5639c6233bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:20.844608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b667ab8-ff85-4c16-be6e-5639c6233bdb None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:20.852358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 630/1250] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:13:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:20.858361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd1e85-3f98-4f34-afcf-ca522c8afee4 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:20.935138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:20.958121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd1e85-3f98-4f34-afcf-ca522c8afee4 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:20.992090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12d0d7-7791-43fe-adfc-d7cc442e2558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:20.992736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12d0d7-7791-43fe-adfc-d7cc442e2558 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:20.993013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12d0d7-7791-43fe-adfc-d7cc442e2558 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:20.993281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12d0d7-7791-43fe-adfc-d7cc442e2558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:21.002889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12d0d7-7791-43fe-adfc-d7cc442e2558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:21.004342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12d0d7-7791-43fe-adfc-d7cc442e2558 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-1a12d0d7-7791-43fe-adfc-d7cc442e2558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:21.019880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:21.021468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:21.022611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:21.022611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:21.022611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:21.022611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 621/1251] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:21.027067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b667ab8-ff85-4c16-be6e-5639c6233bdb None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:21.129691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b667ab8-ff85-4c16-be6e-5639c6233bdb None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:21.162397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bde49-2529-4116-a8de-b2ef9b57ed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:21.162996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bde49-2529-4116-a8de-b2ef9b57ed23 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:21.163274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bde49-2529-4116-a8de-b2ef9b57ed23 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:21.163568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bde49-2529-4116-a8de-b2ef9b57ed23 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:21.164238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bde49-2529-4116-a8de-b2ef9b57ed23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:21.164540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bde49-2529-4116-a8de-b2ef9b57ed23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:21.165895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bde49-2529-4116-a8de-b2ef9b57ed23 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:21.177229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 631/1252] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:21.183474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12d0d7-7791-43fe-adfc-d7cc442e2558 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:21.281327 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12d0d7-7791-43fe-adfc-d7cc442e2558 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:21.307476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:21.315297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7623de1-163a-4eb0-8efd-4d8a0e7ac2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:21.315897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7623de1-163a-4eb0-8efd-4d8a0e7ac2dd None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:21.316177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7623de1-163a-4eb0-8efd-4d8a0e7ac2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:21.316451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7623de1-163a-4eb0-8efd-4d8a0e7ac2dd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:21.323641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7623de1-163a-4eb0-8efd-4d8a0e7ac2dd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:21.326950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7623de1-163a-4eb0-8efd-4d8a0e7ac2dd None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-c7623de1-163a-4eb0-8efd-4d8a0e7ac2dd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:13:21.335935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:21.337059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:21.338272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 622/1253] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:21.343072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bde49-2529-4116-a8de-b2ef9b57ed23 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:21.400732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:21.445347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bde49-2529-4116-a8de-b2ef9b57ed23 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:21.485538 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499fe1a-3c47-4978-acb7-8320b1214030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:21.486140 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499fe1a-3c47-4978-acb7-8320b1214030 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:21.486417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499fe1a-3c47-4978-acb7-8320b1214030 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:21.486687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499fe1a-3c47-4978-acb7-8320b1214030 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:21.487403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499fe1a-3c47-4978-acb7-8320b1214030 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:21.487732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499fe1a-3c47-4978-acb7-8320b1214030 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:21.489232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499fe1a-3c47-4978-acb7-8320b1214030 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:21.505785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 632/1254] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Sep 5 12:13:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:21.511912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7623de1-163a-4eb0-8efd-4d8a0e7ac2dd None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:21.610451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7623de1-163a-4eb0-8efd-4d8a0e7ac2dd None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:21.642354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1b553-3e2f-4681-a4e2-296e4f37075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:21.642942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b553-3e2f-4681-a4e2-296e4f37075e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:21.643223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b553-3e2f-4681-a4e2-296e4f37075e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:21.643492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b553-3e2f-4681-a4e2-296e4f37075e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:21.697707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:21.710055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1b553-3e2f-4681-a4e2-296e4f37075e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:21.711500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1b553-3e2f-4681-a4e2-296e4f37075e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:21.809849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 623/1255] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:13:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:13:21.815983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499fe1a-3c47-4978-acb7-8320b1214030 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:21.914435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499fe1a-3c47-4978-acb7-8320b1214030 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:21.927387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:21.948120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b57fb7-17fb-4633-8184-2c8a486ecb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:21.948679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b57fb7-17fb-4633-8184-2c8a486ecb70 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:21.948957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b57fb7-17fb-4633-8184-2c8a486ecb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:21.949231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b57fb7-17fb-4633-8184-2c8a486ecb70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:21.959551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b57fb7-17fb-4633-8184-2c8a486ecb70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:21.961193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b57fb7-17fb-4633-8184-2c8a486ecb70 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-e5b57fb7-17fb-4633-8184-2c8a486ecb70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:21.975427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:21.976453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:21.977401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:21.977401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:21.977401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:21.977401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 633/1256] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:21.982399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1b553-3e2f-4681-a4e2-296e4f37075e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:22.068985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:22.080742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1b553-3e2f-4681-a4e2-296e4f37075e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:22.114626 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d25f6-80fc-4b49-9bce-d05590ffa521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:22.115227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d25f6-80fc-4b49-9bce-d05590ffa521 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:22.115504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d25f6-80fc-4b49-9bce-d05590ffa521 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:22.115807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d25f6-80fc-4b49-9bce-d05590ffa521 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:22.116436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d25f6-80fc-4b49-9bce-d05590ffa521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:22.116735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d25f6-80fc-4b49-9bce-d05590ffa521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:22.118096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d25f6-80fc-4b49-9bce-d05590ffa521 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:22.128975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 624/1257] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:22.135232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b57fb7-17fb-4633-8184-2c8a486ecb70 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:22.234225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b57fb7-17fb-4633-8184-2c8a486ecb70 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:22.269429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f71159-0d1a-46b7-a039-c1407a34c74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:22.270049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f71159-0d1a-46b7-a039-c1407a34c74f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:22.270328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f71159-0d1a-46b7-a039-c1407a34c74f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:22.270601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f71159-0d1a-46b7-a039-c1407a34c74f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:22.277225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f71159-0d1a-46b7-a039-c1407a34c74f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:22.278696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f71159-0d1a-46b7-a039-c1407a34c74f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f8f71159-0d1a-46b7-a039-c1407a34c74f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:13:22.290686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:22.291687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:22.292595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:13:22.292595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:13:22.292595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:13:22.292595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:13:22.292595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:13:22.292595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:13:22.292595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:22.292595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:22.292595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 634/1258] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:22.297630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d25f6-80fc-4b49-9bce-d05590ffa521 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:22.396528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d25f6-80fc-4b49-9bce-d05590ffa521 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:22.437228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764b940-c624-48b1-ad86-e3d566cdddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:22.437838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764b940-c624-48b1-ad86-e3d566cdddf1 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:22.438118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764b940-c624-48b1-ad86-e3d566cdddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:22.438388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764b940-c624-48b1-ad86-e3d566cdddf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:22.439135 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764b940-c624-48b1-ad86-e3d566cdddf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:22.439493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764b940-c624-48b1-ad86-e3d566cdddf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:22.440997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764b940-c624-48b1-ad86-e3d566cdddf1 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:22.453605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 625/1259] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:13:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:22.459811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f71159-0d1a-46b7-a039-c1407a34c74f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:22.485434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:22.557080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f71159-0d1a-46b7-a039-c1407a34c74f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:22.588707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b07ed8-7707-45b4-a4d5-29591c683535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:22.589414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b07ed8-7707-45b4-a4d5-29591c683535 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:22.589692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b07ed8-7707-45b4-a4d5-29591c683535 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:22.589959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b07ed8-7707-45b4-a4d5-29591c683535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:22.590701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b07ed8-7707-45b4-a4d5-29591c683535 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=90d65372febb414d82843f9dfa56f738) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:22.591004 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b07ed8-7707-45b4-a4d5-29591c683535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:22.592497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b07ed8-7707-45b4-a4d5-29591c683535 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:22.598513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 635/1260] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:13:22 2022] GET /identity/v3/role_assignments?user.id=90d65372febb414d82843f9dfa56f738&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=705907b9bdec47cebf88b868a6a56356 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:24.459967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf241-f27a-4c40-a900-791f122c6fbe None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:24.460462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf241-f27a-4c40-a900-791f122c6fbe None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:24.460462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf241-f27a-4c40-a900-791f122c6fbe None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:24.461825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 626/1261] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:24.467171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc250b21-a663-457f-a9fc-be0fa2161f95 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:24.469270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc250b21-a663-457f-a9fc-be0fa2161f95 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:24.469270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc250b21-a663-457f-a9fc-be0fa2161f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:24.486363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fc250b21-a663-457f-a9fc-be0fa2161f95 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:24.487528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fc250b21-a663-457f-a9fc-be0fa2161f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:24.554149 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 636/1262] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:24.824234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf241-f27a-4c40-a900-791f122c6fbe None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:24.835714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:24.924157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf241-f27a-4c40-a900-791f122c6fbe None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:24.956989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bafb87-9359-4d77-8b45-5de637e7d195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:24.957601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bafb87-9359-4d77-8b45-5de637e7d195 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:24.957879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bafb87-9359-4d77-8b45-5de637e7d195 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:24.958145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bafb87-9359-4d77-8b45-5de637e7d195 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:24.968458 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bafb87-9359-4d77-8b45-5de637e7d195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-88bafb87-9359-4d77-8b45-5de637e7d195 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:13:24.973209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:24.974352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:24.975400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 627/1263] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:13:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:24.980492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc250b21-a663-457f-a9fc-be0fa2161f95 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:24.998652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:25.091963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc250b21-a663-457f-a9fc-be0fa2161f95 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:25.124893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998314e-a1f3-479f-b1eb-bec6e9bb080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:25.125491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998314e-a1f3-479f-b1eb-bec6e9bb080c None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:25.125764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998314e-a1f3-479f-b1eb-bec6e9bb080c None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:25.126031 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998314e-a1f3-479f-b1eb-bec6e9bb080c None admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:25.126687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998314e-a1f3-479f-b1eb-bec6e9bb080c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:25.126983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998314e-a1f3-479f-b1eb-bec6e9bb080c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:25.128360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998314e-a1f3-479f-b1eb-bec6e9bb080c None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:25.136704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 637/1264] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:13:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:25.142641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bafb87-9359-4d77-8b45-5de637e7d195 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:25.211393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:25.243321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bafb87-9359-4d77-8b45-5de637e7d195 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:25.276330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc967e00-7069-44cf-bdd0-961c3b5823a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:25.276946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc967e00-7069-44cf-bdd0-961c3b5823a2 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:25.277225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc967e00-7069-44cf-bdd0-961c3b5823a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:25.277494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc967e00-7069-44cf-bdd0-961c3b5823a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:25.287055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc967e00-7069-44cf-bdd0-961c3b5823a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:25.288526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc967e00-7069-44cf-bdd0-961c3b5823a2 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-cc967e00-7069-44cf-bdd0-961c3b5823a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:25.302459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:25.303962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:25.304927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:25.304927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:25.304927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:25.304927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 628/1265] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:25.309962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998314e-a1f3-479f-b1eb-bec6e9bb080c None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:25.390639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:25.408409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998314e-a1f3-479f-b1eb-bec6e9bb080c None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:25.440594 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707e6f3-a4e6-4ee8-9cfd-780190fe0590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:25.441192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707e6f3-a4e6-4ee8-9cfd-780190fe0590 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:25.441470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707e6f3-a4e6-4ee8-9cfd-780190fe0590 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:25.441741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707e6f3-a4e6-4ee8-9cfd-780190fe0590 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:25.442373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707e6f3-a4e6-4ee8-9cfd-780190fe0590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:25.442671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707e6f3-a4e6-4ee8-9cfd-780190fe0590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:25.444757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707e6f3-a4e6-4ee8-9cfd-780190fe0590 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:25.456140 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 638/1266] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:25.461684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc967e00-7069-44cf-bdd0-961c3b5823a2 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:25.562239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc967e00-7069-44cf-bdd0-961c3b5823a2 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:25.581148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:25.594697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfea0e7-4cca-4add-94fa-fd31794979ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:25.595352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfea0e7-4cca-4add-94fa-fd31794979ac None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:25.595684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfea0e7-4cca-4add-94fa-fd31794979ac None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:25.595958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfea0e7-4cca-4add-94fa-fd31794979ac None admin] PATH_INFO: `/v3/users/placement` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:25.604006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfea0e7-4cca-4add-94fa-fd31794979ac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:25.605436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfea0e7-4cca-4add-94fa-fd31794979ac None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-0cfea0e7-4cca-4add-94fa-fd31794979ac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:13:25.617647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:25.618662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:25.619744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 629/1267] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:25.624653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707e6f3-a4e6-4ee8-9cfd-780190fe0590 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:25.723892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707e6f3-a4e6-4ee8-9cfd-780190fe0590 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:25.758081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e88cc-a5ed-4d4f-b9b4-188901c8af05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:25.758669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e88cc-a5ed-4d4f-b9b4-188901c8af05 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:25.758942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e88cc-a5ed-4d4f-b9b4-188901c8af05 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:25.759210 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e88cc-a5ed-4d4f-b9b4-188901c8af05 None admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:25.759951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e88cc-a5ed-4d4f-b9b4-188901c8af05 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:25.760248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e88cc-a5ed-4d4f-b9b4-188901c8af05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:25.761725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e88cc-a5ed-4d4f-b9b4-188901c8af05 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:25.770135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:25.778980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 639/1268] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Sep 5 12:13:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:25.784676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfea0e7-4cca-4add-94fa-fd31794979ac None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:25.884492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfea0e7-4cca-4add-94fa-fd31794979ac None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:25.918203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb01bb7-3459-4f0f-8f18-94648906c1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:25.918817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb01bb7-3459-4f0f-8f18-94648906c1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:25.919095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb01bb7-3459-4f0f-8f18-94648906c1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:25.919364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb01bb7-3459-4f0f-8f18-94648906c1d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:25.929587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb01bb7-3459-4f0f-8f18-94648906c1d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:25.931040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb01bb7-3459-4f0f-8f18-94648906c1d5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-4eb01bb7-3459-4f0f-8f18-94648906c1d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:25.944647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:25.945784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:25.947083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:25.947083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:25.947083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:25.947083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 630/1269] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 12:13:25.951646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e88cc-a5ed-4d4f-b9b4-188901c8af05 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:26.050822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e88cc-a5ed-4d4f-b9b4-188901c8af05 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:26.085296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6907584-5b5d-42b1-9782-39536c006bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:26.085900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6907584-5b5d-42b1-9782-39536c006bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:26.086175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6907584-5b5d-42b1-9782-39536c006bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:26.086444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6907584-5b5d-42b1-9782-39536c006bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:26.087084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6907584-5b5d-42b1-9782-39536c006bf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:26.087380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6907584-5b5d-42b1-9782-39536c006bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:26.088773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6907584-5b5d-42b1-9782-39536c006bf4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:26.105490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 640/1270] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:26.106684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb01bb7-3459-4f0f-8f18-94648906c1d5 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:26.137838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:26.203383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb01bb7-3459-4f0f-8f18-94648906c1d5 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:26.238644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09662be8-c594-4ebb-9248-372ae8a23ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:26.239266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09662be8-c594-4ebb-9248-372ae8a23ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:26.239564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09662be8-c594-4ebb-9248-372ae8a23ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:26.239843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09662be8-c594-4ebb-9248-372ae8a23ae9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:26.246765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09662be8-c594-4ebb-9248-372ae8a23ae9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:26.248257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09662be8-c594-4ebb-9248-372ae8a23ae9 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-09662be8-c594-4ebb-9248-372ae8a23ae9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:13:26.259992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:26.261043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:26.262329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:13:26.262329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:13:26.262329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:13:26.262329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:13:26.262329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:13:26.262329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:13:26.262329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:26.262329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:26.262329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 631/1271] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:26.267029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6907584-5b5d-42b1-9782-39536c006bf4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:26.327363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:26.368862 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6907584-5b5d-42b1-9782-39536c006bf4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:26.401753 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4ae9f-8674-4407-8e38-5c84179f427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:26.402349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4ae9f-8674-4407-8e38-5c84179f427a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:26.402626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4ae9f-8674-4407-8e38-5c84179f427a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:26.402896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4ae9f-8674-4407-8e38-5c84179f427a None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:26.403694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4ae9f-8674-4407-8e38-5c84179f427a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:26.403992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4ae9f-8674-4407-8e38-5c84179f427a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:26.405404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4ae9f-8674-4407-8e38-5c84179f427a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:26.417011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 641/1272] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:26.423096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09662be8-c594-4ebb-9248-372ae8a23ae9 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:26.514401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:26.523321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09662be8-c594-4ebb-9248-372ae8a23ae9 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:26.555066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a655b-e428-4e3f-b68e-605dc67dffd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:26.555736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a655b-e428-4e3f-b68e-605dc67dffd6 None admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:26.556014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a655b-e428-4e3f-b68e-605dc67dffd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:26.556284 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a655b-e428-4e3f-b68e-605dc67dffd6 None admin] PATH_INFO: `/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/90d65372febb414d82843f9dfa56f738/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:26.558130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a655b-e428-4e3f-b68e-605dc67dffd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c48ceeacd6453ca9af78ca41554230, user_id=90d65372febb414d82843f9dfa56f738, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:26.559569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a655b-e428-4e3f-b68e-605dc67dffd6 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:26.665062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 632/1273] 10.222.0.62 () {58 vars in 1439 bytes} [Mon Sep 5 12:13:26 2022] PUT /identity/v3/projects/f2c48ceeacd6453ca9af78ca41554230/users/90d65372febb414d82843f9dfa56f738/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:13:28.560908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b4eae-1911-4ba5-95b2-f07906fb025a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:28.561455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b4eae-1911-4ba5-95b2-f07906fb025a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:28.561455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b4eae-1911-4ba5-95b2-f07906fb025a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:28.562775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 642/1274] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:28.567997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1fda6-cdf9-458d-a74c-818a993dba79 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:28.568400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1fda6-cdf9-458d-a74c-818a993dba79 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:28.568535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1fda6-cdf9-458d-a74c-818a993dba79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:28.586892 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-2de1fda6-cdf9-458d-a74c-818a993dba79 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:28.588108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2de1fda6-cdf9-458d-a74c-818a993dba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:28.653352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 633/1275] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:28.919390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b4eae-1911-4ba5-95b2-f07906fb025a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:29.021231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b4eae-1911-4ba5-95b2-f07906fb025a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:29.039250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:29.055799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a85550-ecfd-42c8-be1b-6f49320fb920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:29.056423 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a85550-ecfd-42c8-be1b-6f49320fb920 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:29.056698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a85550-ecfd-42c8-be1b-6f49320fb920 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:29.056996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a85550-ecfd-42c8-be1b-6f49320fb920 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:29.067076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a85550-ecfd-42c8-be1b-6f49320fb920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-10a85550-ecfd-42c8-be1b-6f49320fb920 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 12:13:29.071868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:29.073012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:29.074003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 643/1276] 10.222.0.62 () {56 vars in 1096 bytes} [Mon Sep 5 12:13:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:29.078938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1fda6-cdf9-458d-a74c-818a993dba79 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:29.158968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:29.177240 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1fda6-cdf9-458d-a74c-818a993dba79 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:29.209923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056c42-6b91-486a-9b38-4dcbf6caed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:29.210526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056c42-6b91-486a-9b38-4dcbf6caed46 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:29.210804 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056c42-6b91-486a-9b38-4dcbf6caed46 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:29.211075 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056c42-6b91-486a-9b38-4dcbf6caed46 None admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:29.211755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9056c42-6b91-486a-9b38-4dcbf6caed46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:29.212059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9056c42-6b91-486a-9b38-4dcbf6caed46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:29.213470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9056c42-6b91-486a-9b38-4dcbf6caed46 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:29.221225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 634/1277] 10.222.0.62 () {56 vars in 1099 bytes} [Mon Sep 5 12:13:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:29.227463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a85550-ecfd-42c8-be1b-6f49320fb920 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:29.327562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a85550-ecfd-42c8-be1b-6f49320fb920 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:29.362246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde82e1-3e29-4574-8d6c-30e8cf709762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:29.363059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde82e1-3e29-4574-8d6c-30e8cf709762 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:29.363175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde82e1-3e29-4574-8d6c-30e8cf709762 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:29.363403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde82e1-3e29-4574-8d6c-30e8cf709762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:29.373803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde82e1-3e29-4574-8d6c-30e8cf709762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:29.375233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde82e1-3e29-4574-8d6c-30e8cf709762 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-3bde82e1-3e29-4574-8d6c-30e8cf709762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:29.390380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:29.394589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:29.395755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:29.395755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:29.395755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:29.395755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 644/1278] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:29.398159 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056c42-6b91-486a-9b38-4dcbf6caed46 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:29.495646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056c42-6b91-486a-9b38-4dcbf6caed46 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:29.528691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75bf52-75b4-4a66-92ef-ea22d5d1f0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:29.529288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75bf52-75b4-4a66-92ef-ea22d5d1f0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:29.529567 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75bf52-75b4-4a66-92ef-ea22d5d1f0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:29.529840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75bf52-75b4-4a66-92ef-ea22d5d1f0d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:29.530472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75bf52-75b4-4a66-92ef-ea22d5d1f0d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:29.530774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75bf52-75b4-4a66-92ef-ea22d5d1f0d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:29.532168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75bf52-75b4-4a66-92ef-ea22d5d1f0d0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:29.543407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 635/1279] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:29.549061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde82e1-3e29-4574-8d6c-30e8cf709762 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:29.580915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:29.649559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde82e1-3e29-4574-8d6c-30e8cf709762 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:29.682785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e7fca-7994-454b-b6ce-2b4afb4e7e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:29.683407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e7fca-7994-454b-b6ce-2b4afb4e7e6e None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:29.683712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e7fca-7994-454b-b6ce-2b4afb4e7e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:29.684000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e7fca-7994-454b-b6ce-2b4afb4e7e6e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:29.691526 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e7fca-7994-454b-b6ce-2b4afb4e7e6e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:29.692979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e7fca-7994-454b-b6ce-2b4afb4e7e6e None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-363e7fca-7994-454b-b6ce-2b4afb4e7e6e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 12:13:29.704781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:29.705928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:29.707164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 645/1280] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:29.711831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75bf52-75b4-4a66-92ef-ea22d5d1f0d0 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:29.715528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:29.809239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75bf52-75b4-4a66-92ef-ea22d5d1f0d0 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:29.843852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac938f-2b86-464e-875c-a4af6197fbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:29.844445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac938f-2b86-464e-875c-a4af6197fbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:29.844723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac938f-2b86-464e-875c-a4af6197fbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:29.844995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac938f-2b86-464e-875c-a4af6197fbc1 None admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:29.845705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac938f-2b86-464e-875c-a4af6197fbc1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:29.846004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac938f-2b86-464e-875c-a4af6197fbc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:29.847421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac938f-2b86-464e-875c-a4af6197fbc1 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:29.865288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 636/1281] 10.222.0.62 () {56 vars in 1143 bytes} [Mon Sep 5 12:13:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:29.871594 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e7fca-7994-454b-b6ce-2b4afb4e7e6e None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:29.960233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:29.969240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e7fca-7994-454b-b6ce-2b4afb4e7e6e None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:30.001891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece874e-d1e4-49fb-b3cf-088b4fb395f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:30.002493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece874e-d1e4-49fb-b3cf-088b4fb395f8 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:30.002772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece874e-d1e4-49fb-b3cf-088b4fb395f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:30.003042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece874e-d1e4-49fb-b3cf-088b4fb395f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:30.071176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece874e-d1e4-49fb-b3cf-088b4fb395f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:30.072612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece874e-d1e4-49fb-b3cf-088b4fb395f8 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:30.160745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:30.171417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 646/1282] 10.222.0.62 () {58 vars in 1282 bytes} [Mon Sep 5 12:13:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 12:13:30.177421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac938f-2b86-464e-875c-a4af6197fbc1 None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:30.237948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:30.275281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac938f-2b86-464e-875c-a4af6197fbc1 None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:30.308704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b320c-858d-40ff-91d8-9cc0c8351a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:30.309321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b320c-858d-40ff-91d8-9cc0c8351a6f None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:30.309600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b320c-858d-40ff-91d8-9cc0c8351a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:30.309954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b320c-858d-40ff-91d8-9cc0c8351a6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:30.319435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b320c-858d-40ff-91d8-9cc0c8351a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:30.320917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b320c-858d-40ff-91d8-9cc0c8351a6f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-c21b320c-858d-40ff-91d8-9cc0c8351a6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 12:13:30.336089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:30.337271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 12:13:30.338332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 12:13:30.338332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 12:13:30.338332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:30.338332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 637/1283] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:30.342840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece874e-d1e4-49fb-b3cf-088b4fb395f8 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:30.439738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece874e-d1e4-49fb-b3cf-088b4fb395f8 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:30.474052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dc372-5061-4e7d-a17f-d2b143ddad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:30.474659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dc372-5061-4e7d-a17f-d2b143ddad8f None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:30.474939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dc372-5061-4e7d-a17f-d2b143ddad8f None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:30.475254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dc372-5061-4e7d-a17f-d2b143ddad8f None admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:30.476459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dc372-5061-4e7d-a17f-d2b143ddad8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:30.476760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dc372-5061-4e7d-a17f-d2b143ddad8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:30.478208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dc372-5061-4e7d-a17f-d2b143ddad8f None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:30.489758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 647/1284] 10.222.0.62 () {56 vars in 1109 bytes} [Mon Sep 5 12:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:30.495520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b320c-858d-40ff-91d8-9cc0c8351a6f None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:30.597129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b320c-858d-40ff-91d8-9cc0c8351a6f None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:30.611241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:30.630549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a5d31-a189-45f6-ba81-071bb3ebe23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:30.631168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5d31-a189-45f6-ba81-071bb3ebe23c None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:30.631512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5d31-a189-45f6-ba81-071bb3ebe23c None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:30.638787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5d31-a189-45f6-ba81-071bb3ebe23c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:30.638787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a5d31-a189-45f6-ba81-071bb3ebe23c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:30.639751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a5d31-a189-45f6-ba81-071bb3ebe23c None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application [None req-a29a5d31-a189-45f6-ba81-071bb3ebe23c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 12:13:30.651276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:30.652391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:30.653334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 12:13:30.653334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 12:13:30.653334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 12:13:30.653334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 12:13:30.653334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 12:13:30.653334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 12:13:30.653334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 12:13:30.653334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: ERROR keystone.server.flask.application Sep 05 12:13:30.653334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 638/1285] 10.222.0.62 () {56 vars in 1111 bytes} [Mon Sep 5 12:13:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:30.658225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dc372-5061-4e7d-a17f-d2b143ddad8f None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:30.707308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:30.756107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dc372-5061-4e7d-a17f-d2b143ddad8f None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:30.788983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e0fbb-5640-432c-8f08-036bdc1e3b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:30.789576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e0fbb-5640-432c-8f08-036bdc1e3b60 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:30.789850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e0fbb-5640-432c-8f08-036bdc1e3b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:30.790116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e0fbb-5640-432c-8f08-036bdc1e3b60 None admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:30.790796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e0fbb-5640-432c-8f08-036bdc1e3b60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:30.791095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e0fbb-5640-432c-8f08-036bdc1e3b60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:30.792529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e0fbb-5640-432c-8f08-036bdc1e3b60 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:30.804597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 648/1286] 10.222.0.62 () {56 vars in 1148 bytes} [Mon Sep 5 12:13:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:30.810748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a5d31-a189-45f6-ba81-071bb3ebe23c None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:30.910070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a5d31-a189-45f6-ba81-071bb3ebe23c None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:30.941947 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eabdf3-062f-4d88-a82e-5ea1e6c8d3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:30.942540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eabdf3-062f-4d88-a82e-5ea1e6c8d3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:30.942818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eabdf3-062f-4d88-a82e-5ea1e6c8d3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:30.943092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eabdf3-062f-4d88-a82e-5ea1e6c8d3c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:30.944410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eabdf3-062f-4d88-a82e-5ea1e6c8d3c0 None admin] RBAC: Adding query filter params (role.id=705907b9bdec47cebf88b868a6a56356, scope.project.id=f2c48ceeacd6453ca9af78ca41554230, user.id=90d65372febb414d82843f9dfa56f738) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:30.944824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eabdf3-062f-4d88-a82e-5ea1e6c8d3c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:30.946328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eabdf3-062f-4d88-a82e-5ea1e6c8d3c0 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:30.952020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 639/1287] 10.222.0.62 () {56 vars in 1374 bytes} [Mon Sep 5 12:13:30 2022] GET /identity/v3/role_assignments?user.id=90d65372febb414d82843f9dfa56f738&scope.project.id=f2c48ceeacd6453ca9af78ca41554230&role.id=705907b9bdec47cebf88b868a6a56356 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:32.860578 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a0e7-0c82-41d0-aa80-5cb2327d38ed None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:32.861158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a0e7-0c82-41d0-aa80-5cb2327d38ed None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:32.861158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a0e7-0c82-41d0-aa80-5cb2327d38ed None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:32.862453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 649/1288] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:32.868051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ff9ec-f245-4e74-af53-39e588dc6a40 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:32.868483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ff9ec-f245-4e74-af53-39e588dc6a40 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:32.868590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ff9ec-f245-4e74-af53-39e588dc6a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:32.887367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-023ff9ec-f245-4e74-af53-39e588dc6a40 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:32.888581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-023ff9ec-f245-4e74-af53-39e588dc6a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:32.916657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:32.953455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 640/1289] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:33.231456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191a0e7-0c82-41d0-aa80-5cb2327d38ed None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:33.332441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191a0e7-0c82-41d0-aa80-5cb2327d38ed None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:33.342922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:33.366779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71909ce-9c0f-4654-8f11-249b984c0cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:33.367419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71909ce-9c0f-4654-8f11-249b984c0cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:33.367723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71909ce-9c0f-4654-8f11-249b984c0cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:33.367996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71909ce-9c0f-4654-8f11-249b984c0cd4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:33.368571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71909ce-9c0f-4654-8f11-249b984c0cd4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:33.369947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71909ce-9c0f-4654-8f11-249b984c0cd4 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-f71909ce-9c0f-4654-8f11-249b984c0cd4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:33.378677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:33.379798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:33.381073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 650/1290] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Sep 5 12:13:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:33.385779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ff9ec-f245-4e74-af53-39e588dc6a40 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:33.481933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ff9ec-f245-4e74-af53-39e588dc6a40 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:33.515233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f88667-faf9-42a3-af44-ff86f68491b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:33.515852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f88667-faf9-42a3-af44-ff86f68491b5 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:33.516125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f88667-faf9-42a3-af44-ff86f68491b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:33.516393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f88667-faf9-42a3-af44-ff86f68491b5 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:33.517009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f88667-faf9-42a3-af44-ff86f68491b5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:33.517304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f88667-faf9-42a3-af44-ff86f68491b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:33.518652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f88667-faf9-42a3-af44-ff86f68491b5 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:33.523832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 641/1291] 10.222.0.62 () {56 vars in 1116 bytes} [Mon Sep 5 12:13:33 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:33.529551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71909ce-9c0f-4654-8f11-249b984c0cd4 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:33.630183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71909ce-9c0f-4654-8f11-249b984c0cd4 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:33.662660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b13ded-ddc7-4a57-ac2e-c209d8801d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:33.663856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13ded-ddc7-4a57-ac2e-c209d8801d32 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:33.664137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13ded-ddc7-4a57-ac2e-c209d8801d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:33.664409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13ded-ddc7-4a57-ac2e-c209d8801d32 None admin] PATH_INFO: `/v3/services` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:33.665033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b13ded-ddc7-4a57-ac2e-c209d8801d32 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:33.665326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b13ded-ddc7-4a57-ac2e-c209d8801d32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:33.666682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b13ded-ddc7-4a57-ac2e-c209d8801d32 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:33.670992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 651/1292] 10.222.0.62 () {56 vars in 1116 bytes} [Mon Sep 5 12:13:33 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:35.540584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1df5e-bbda-4c66-a95b-19b2200dbfc4 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:35.541128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1df5e-bbda-4c66-a95b-19b2200dbfc4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:35.541128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1df5e-bbda-4c66-a95b-19b2200dbfc4 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:35.542468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 642/1293] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:35.547618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613703-2a32-4bed-8944-9553916f392a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:35.551769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613703-2a32-4bed-8944-9553916f392a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:35.551769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613703-2a32-4bed-8944-9553916f392a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:35.566706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f7613703-2a32-4bed-8944-9553916f392a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:35.567915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f7613703-2a32-4bed-8944-9553916f392a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:35.596702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:35.634238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 652/1294] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:35.900745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1df5e-bbda-4c66-a95b-19b2200dbfc4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:35.931933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:36.001621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1df5e-bbda-4c66-a95b-19b2200dbfc4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:36.034452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4eb91c-311b-411a-869a-b145622c3e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:36.035046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eb91c-311b-411a-869a-b145622c3e1f None admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:36.035323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eb91c-311b-411a-869a-b145622c3e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:36.035619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eb91c-311b-411a-869a-b145622c3e1f None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:36.036254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4eb91c-311b-411a-869a-b145622c3e1f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:36.037607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4eb91c-311b-411a-869a-b145622c3e1f None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:36.114813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 643/1295] 10.222.0.62 () {60 vars in 1141 bytes} [Mon Sep 5 12:13:35 2022] POST /identity/v3/services => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:13:38.075679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97b279-121a-4ac1-a164-1d4f43d125fb None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:38.076171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97b279-121a-4ac1-a164-1d4f43d125fb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:38.076265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97b279-121a-4ac1-a164-1d4f43d125fb None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:38.077540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 653/1296] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:38.082745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea38d-8c37-4bb6-89b1-42f56b7b4905 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:38.083156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea38d-8c37-4bb6-89b1-42f56b7b4905 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:38.083276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea38d-8c37-4bb6-89b1-42f56b7b4905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:38.102235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-f63ea38d-8c37-4bb6-89b1-42f56b7b4905 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:38.103408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f63ea38d-8c37-4bb6-89b1-42f56b7b4905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:38.174514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 644/1297] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:38.441443 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97b279-121a-4ac1-a164-1d4f43d125fb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:38.540818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97b279-121a-4ac1-a164-1d4f43d125fb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:38.573566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424db62e-9d23-436e-ac39-569658639f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:38.574184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424db62e-9d23-436e-ac39-569658639f6a None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:38.574458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424db62e-9d23-436e-ac39-569658639f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:38.574724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424db62e-9d23-436e-ac39-569658639f6a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:38.575294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424db62e-9d23-436e-ac39-569658639f6a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:38.576718 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424db62e-9d23-436e-ac39-569658639f6a None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-424db62e-9d23-436e-ac39-569658639f6a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:38.585347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:38.586560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:38.587987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 654/1298] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Sep 5 12:13:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:38.593675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea38d-8c37-4bb6-89b1-42f56b7b4905 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:38.612671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:38.691275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea38d-8c37-4bb6-89b1-42f56b7b4905 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:38.723346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d433f4-9c8a-41be-8226-1e42bd496a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:38.723916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d433f4-9c8a-41be-8226-1e42bd496a8e None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:38.724193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d433f4-9c8a-41be-8226-1e42bd496a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:38.724462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d433f4-9c8a-41be-8226-1e42bd496a8e None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:38.725079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d433f4-9c8a-41be-8226-1e42bd496a8e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:38.725378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d433f4-9c8a-41be-8226-1e42bd496a8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:38.727359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d433f4-9c8a-41be-8226-1e42bd496a8e None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:38.732246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 645/1299] 10.222.0.62 () {56 vars in 1116 bytes} [Mon Sep 5 12:13:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:38.738445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424db62e-9d23-436e-ac39-569658639f6a None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:38.769014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:38.836326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424db62e-9d23-436e-ac39-569658639f6a None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:38.871151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7299f-4b0f-40d2-866b-cce5c29b8bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:38.871790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7299f-4b0f-40d2-866b-cce5c29b8bea None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:38.872065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7299f-4b0f-40d2-866b-cce5c29b8bea None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:38.872334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7299f-4b0f-40d2-866b-cce5c29b8bea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:38.873981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7299f-4b0f-40d2-866b-cce5c29b8bea None admin] RBAC: Adding query filter params (interface=public, service_id=22d975ef9dae42eea5218775d7e80c9d, region_id=RegionOne) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:38.874299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7299f-4b0f-40d2-866b-cce5c29b8bea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:38.875811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7299f-4b0f-40d2-866b-cce5c29b8bea None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:38.880887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 655/1300] 10.222.0.62 () {56 vars in 1251 bytes} [Mon Sep 5 12:13:38 2022] GET /identity/v3/endpoints?service_id=22d975ef9dae42eea5218775d7e80c9d&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:38.886543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d433f4-9c8a-41be-8226-1e42bd496a8e None admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:38.967375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:38.985974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d433f4-9c8a-41be-8226-1e42bd496a8e None admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:39.019671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b83a74-9248-4d5c-a2b1-a69e2cb44fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:39.020264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b83a74-9248-4d5c-a2b1-a69e2cb44fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:39.020600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b83a74-9248-4d5c-a2b1-a69e2cb44fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:39.020875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b83a74-9248-4d5c-a2b1-a69e2cb44fd3 None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:39.021471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b83a74-9248-4d5c-a2b1-a69e2cb44fd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:39.023332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b83a74-9248-4d5c-a2b1-a69e2cb44fd3 None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:39.030098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 646/1301] 10.222.0.62 () {56 vars in 1087 bytes} [Mon Sep 5 12:13:38 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:13:40.899571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95978c10-4b9d-4177-8b84-ea1a4487a23a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:40.900132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95978c10-4b9d-4177-8b84-ea1a4487a23a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:40.900132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95978c10-4b9d-4177-8b84-ea1a4487a23a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:40.901568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 656/1302] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:13:40.906844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa085542-5c79-46c6-b34c-acc3f3985646 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:40.907233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa085542-5c79-46c6-b34c-acc3f3985646 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:40.907409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa085542-5c79-46c6-b34c-acc3f3985646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:40.925387 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-aa085542-5c79-46c6-b34c-acc3f3985646 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:13:40.929976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-aa085542-5c79-46c6-b34c-acc3f3985646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:13:40.993392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 647/1303] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:13:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 12:13:41.261636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95978c10-4b9d-4177-8b84-ea1a4487a23a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:41.357211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:41.364085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95978c10-4b9d-4177-8b84-ea1a4487a23a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:41.396866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8726e-8c66-4676-a0bd-9b453550f8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:41.397481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8726e-8c66-4676-a0bd-9b453550f8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:41.398352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8726e-8c66-4676-a0bd-9b453550f8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:41.398617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8726e-8c66-4676-a0bd-9b453550f8f3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:41.399186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8726e-8c66-4676-a0bd-9b453550f8f3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:41.400590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8726e-8c66-4676-a0bd-9b453550f8f3 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application [None req-19d8726e-8c66-4676-a0bd-9b453550f8f3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 12:13:41.409434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 12:13:41.410622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application with Lock( Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self._enter() Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application return self.creator() Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application created_value = creator( Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: ERROR keystone.server.flask.application Sep 05 12:13:41.411585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 657/1304] 10.222.0.62 () {56 vars in 1117 bytes} [Mon Sep 5 12:13:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 12:13:41.419152 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa085542-5c79-46c6-b34c-acc3f3985646 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:41.492279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:41.514947 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa085542-5c79-46c6-b34c-acc3f3985646 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:41.548594 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f659b-37f1-468b-b210-37f2577e8c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:41.549189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f659b-37f1-468b-b210-37f2577e8c3d None admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:41.549463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f659b-37f1-468b-b210-37f2577e8c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:41.549728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f659b-37f1-468b-b210-37f2577e8c3d None admin] PATH_INFO: `/v3/services` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:41.550345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f659b-37f1-468b-b210-37f2577e8c3d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:13:41.550682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f659b-37f1-468b-b210-37f2577e8c3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:41.552342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f659b-37f1-468b-b210-37f2577e8c3d None admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:41.558006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 648/1305] 10.222.0.62 () {56 vars in 1116 bytes} [Mon Sep 5 12:13:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:13:41.563975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8726e-8c66-4676-a0bd-9b453550f8f3 None admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:13:41.660771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8726e-8c66-4676-a0bd-9b453550f8f3 None admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:13:41.692785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676483e-0cd3-4005-8dc3-0a2373170779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:13:41.693372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676483e-0cd3-4005-8dc3-0a2373170779 None admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:13:41.693644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676483e-0cd3-4005-8dc3-0a2373170779 None admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:13:41.693913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676483e-0cd3-4005-8dc3-0a2373170779 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:13:41.694559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676483e-0cd3-4005-8dc3-0a2373170779 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:13:41.696009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676483e-0cd3-4005-8dc3-0a2373170779 None admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:13:41.828279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 658/1306] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:13:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:13:51.555002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=30 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:13:51.823805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:36.864834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a42961-ab40-4260-9f53-4da9261de4a8 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:36.865416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a42961-ab40-4260-9f53-4da9261de4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:36.865416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a42961-ab40-4260-9f53-4da9261de4a8 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:36.866939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 649/1307] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:36.877728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4706ae6-6831-4d58-8c26-ba5aefaf3ea9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:36.878998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4706ae6-6831-4d58-8c26-ba5aefaf3ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:36.880407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4706ae6-6831-4d58-8c26-ba5aefaf3ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:37.019059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f4706ae6-6831-4d58-8c26-ba5aefaf3ea9 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:37.020577 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f4706ae6-6831-4d58-8c26-ba5aefaf3ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:37.176053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 659/1308] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:37.489225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a38fa7-69bc-4998-8e70-38ff69d5d567 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:37.489556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a38fa7-69bc-4998-8e70-38ff69d5d567 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:37.490063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a38fa7-69bc-4998-8e70-38ff69d5d567 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:37.491823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 650/1309] 10.222.0.62 () {54 vars in 965 bytes} [Mon Sep 5 12:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:37.499758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8d1c0-e1c8-41fd-9744-94b0d3f71ebd None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:37.500124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8d1c0-e1c8-41fd-9744-94b0d3f71ebd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:37.500342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8d1c0-e1c8-41fd-9744-94b0d3f71ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:37.596664 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-80e8d1c0-e1c8-41fd-9744-94b0d3f71ebd None None] MFA Rules not processed for user `9a89a2e6aeb4495a84a61bd82b02bfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:37.597906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-80e8d1c0-e1c8-41fd-9744-94b0d3f71ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:37.623876 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:37.692119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 660/1310] 10.222.0.62 () {58 vars in 1062 bytes} [Mon Sep 5 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:37.701049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a38fa7-69bc-4998-8e70-38ff69d5d567 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:37.702965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f2a38fa7-69bc-4998-8e70-38ff69d5d567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:37.842363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a38fa7-69bc-4998-8e70-38ff69d5d567 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:37.925980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37225a4-2e12-4731-9586-dc3e736402f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:37.926646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37225a4-2e12-4731-9586-dc3e736402f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:37.926740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37225a4-2e12-4731-9586-dc3e736402f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:37.926997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37225a4-2e12-4731-9586-dc3e736402f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:37.928645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a37225a4-2e12-4731-9586-dc3e736402f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:38.010052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:38.085869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37225a4-2e12-4731-9586-dc3e736402f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:38.088019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37225a4-2e12-4731-9586-dc3e736402f3 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:38.109137 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 651/1311] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:40.415909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc09d9b-9c8a-4d29-8f03-e95024271a4a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:40.416461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc09d9b-9c8a-4d29-8f03-e95024271a4a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:40.416461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc09d9b-9c8a-4d29-8f03-e95024271a4a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:40.419890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 661/1312] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:20:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:40.423822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38860d-d782-4e9a-8415-2c3a02393c1a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:40.424237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38860d-d782-4e9a-8415-2c3a02393c1a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:40.424491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38860d-d782-4e9a-8415-2c3a02393c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:40.511945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8f38860d-d782-4e9a-8415-2c3a02393c1a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:40.513167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8f38860d-d782-4e9a-8415-2c3a02393c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:40.617201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 652/1313] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:40.900167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc09d9b-9c8a-4d29-8f03-e95024271a4a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:40.901810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3cc09d9b-9c8a-4d29-8f03-e95024271a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:41.019005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc09d9b-9c8a-4d29-8f03-e95024271a4a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:41.100607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:41.103736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11bb4e-681b-46eb-8a7e-7327677cae33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:41.104347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11bb4e-681b-46eb-8a7e-7327677cae33 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:41.104623 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11bb4e-681b-46eb-8a7e-7327677cae33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:41.104892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11bb4e-681b-46eb-8a7e-7327677cae33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:41.106530 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9c11bb4e-681b-46eb-8a7e-7327677cae33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:41.231175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11bb4e-681b-46eb-8a7e-7327677cae33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:41.232584 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11bb4e-681b-46eb-8a7e-7327677cae33 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:41.256204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 662/1314] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:43.753768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7621b94-da56-4138-bc40-71688d106cf8 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:43.754351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7621b94-da56-4138-bc40-71688d106cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:43.754351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7621b94-da56-4138-bc40-71688d106cf8 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:43.755716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 653/1315] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:43.761148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f137d-8c1a-4b01-a9fd-6f894e8938ed None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:43.761540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f137d-8c1a-4b01-a9fd-6f894e8938ed None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:43.761701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f137d-8c1a-4b01-a9fd-6f894e8938ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:43.834892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7c2f137d-8c1a-4b01-a9fd-6f894e8938ed None None] MFA Rules not processed for user `6b6757539b0548b8a38441652f18bce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:43.837107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7c2f137d-8c1a-4b01-a9fd-6f894e8938ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:43.912633 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:43.992262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 663/1316] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:20:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:44.294945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301509c-c079-4b73-b36f-29a4dd471f80 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:44.295222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301509c-c079-4b73-b36f-29a4dd471f80 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:44.295494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301509c-c079-4b73-b36f-29a4dd471f80 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:44.296864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 654/1317] 10.222.0.62 () {54 vars in 965 bytes} [Mon Sep 5 12:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:44.303170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b58cbb-2dc3-4a39-b2ea-d35fe0645601 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:44.303382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b58cbb-2dc3-4a39-b2ea-d35fe0645601 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:44.303696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b58cbb-2dc3-4a39-b2ea-d35fe0645601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:44.381697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-03b58cbb-2dc3-4a39-b2ea-d35fe0645601 None None] MFA Rules not processed for user `9a89a2e6aeb4495a84a61bd82b02bfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:44.382930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-03b58cbb-2dc3-4a39-b2ea-d35fe0645601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:44.461744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 664/1318] 10.222.0.62 () {58 vars in 1062 bytes} [Mon Sep 5 12:20:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:20:44.471381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301509c-c079-4b73-b36f-29a4dd471f80 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:44.472936 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4301509c-c079-4b73-b36f-29a4dd471f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:44.483418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:44.588380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301509c-c079-4b73-b36f-29a4dd471f80 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:44.671356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d5af0-b930-490d-87de-b5b6613ab24c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:44.672000 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d5af0-b930-490d-87de-b5b6613ab24c service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:44.672286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d5af0-b930-490d-87de-b5b6613ab24c service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:44.672553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d5af0-b930-490d-87de-b5b6613ab24c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:44.674174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-bb3d5af0-b930-490d-87de-b5b6613ab24c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:44.739357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:44.790322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d5af0-b930-490d-87de-b5b6613ab24c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:44.791849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d5af0-b930-490d-87de-b5b6613ab24c service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:44.810119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 655/1319] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:48.020502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96551a7b-d87e-41c3-a5ff-65865aeae5ce None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:48.021038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96551a7b-d87e-41c3-a5ff-65865aeae5ce None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:48.021157 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96551a7b-d87e-41c3-a5ff-65865aeae5ce None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:48.022434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 665/1320] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:48.028435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ecf03-6de7-41f7-963b-2a54537aa4bd None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:48.028876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ecf03-6de7-41f7-963b-2a54537aa4bd None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:48.028982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ecf03-6de7-41f7-963b-2a54537aa4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:48.096528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-487ecf03-6de7-41f7-963b-2a54537aa4bd None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:48.097744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-487ecf03-6de7-41f7-963b-2a54537aa4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:48.194371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 656/1321] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:20:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:48.490104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96551a7b-d87e-41c3-a5ff-65865aeae5ce None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:48.491673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-96551a7b-d87e-41c3-a5ff-65865aeae5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:48.545042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:48.604002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96551a7b-d87e-41c3-a5ff-65865aeae5ce None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:48.687821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73313b3b-4c29-4f09-9130-2e3e7ff1d0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:48.688437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73313b3b-4c29-4f09-9130-2e3e7ff1d0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:48.688708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73313b3b-4c29-4f09-9130-2e3e7ff1d0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:48.689028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73313b3b-4c29-4f09-9130-2e3e7ff1d0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:48.690764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-73313b3b-4c29-4f09-9130-2e3e7ff1d0d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:48.793147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:48.807670 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73313b3b-4c29-4f09-9130-2e3e7ff1d0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:48.809107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73313b3b-4c29-4f09-9130-2e3e7ff1d0d2 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:48.827918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 666/1322] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:51.953360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d4b23-bca3-48b6-8a72-ee1d56e06ea5 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:51.953912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d4b23-bca3-48b6-8a72-ee1d56e06ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:51.953912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d4b23-bca3-48b6-8a72-ee1d56e06ea5 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:51.955242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 657/1323] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:51.960723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b380d-2424-43e2-a9ee-3904093696be None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:51.961204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b380d-2424-43e2-a9ee-3904093696be None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:51.961301 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b380d-2424-43e2-a9ee-3904093696be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:52.035031 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0b8b380d-2424-43e2-a9ee-3904093696be None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:52.036299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0b8b380d-2424-43e2-a9ee-3904093696be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:52.162737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 667/1324] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:20:52.449818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d4b23-bca3-48b6-8a72-ee1d56e06ea5 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:52.451444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-419d4b23-bca3-48b6-8a72-ee1d56e06ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:52.481346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:52.613431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d4b23-bca3-48b6-8a72-ee1d56e06ea5 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:52.706458 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68a1d0-b55f-4bce-8157-dabb93e09b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:52.707412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68a1d0-b55f-4bce-8157-dabb93e09b09 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:52.707948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68a1d0-b55f-4bce-8157-dabb93e09b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:52.708412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68a1d0-b55f-4bce-8157-dabb93e09b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:52.711250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4c68a1d0-b55f-4bce-8157-dabb93e09b09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:52.815688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:52.856018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68a1d0-b55f-4bce-8157-dabb93e09b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:52.857549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68a1d0-b55f-4bce-8157-dabb93e09b09 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:52.877823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 658/1325] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:54.904039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a2afe-cab5-4138-ab0f-45f18443adeb None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.904605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a2afe-cab5-4138-ab0f-45f18443adeb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.904605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a2afe-cab5-4138-ab0f-45f18443adeb None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.905897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 668/1326] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:54.911217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab99d3-c0a3-4a7a-b2a9-3ac2603fd58f None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:54.911710 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab99d3-c0a3-4a7a-b2a9-3ac2603fd58f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:54.911804 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab99d3-c0a3-4a7a-b2a9-3ac2603fd58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:54.982211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-14ab99d3-c0a3-4a7a-b2a9-3ac2603fd58f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:54.983289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-14ab99d3-c0a3-4a7a-b2a9-3ac2603fd58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:55.089775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 659/1327] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:55.375471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a2afe-cab5-4138-ab0f-45f18443adeb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:55.377153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-819a2afe-cab5-4138-ab0f-45f18443adeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:55.441810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:55.489566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a2afe-cab5-4138-ab0f-45f18443adeb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:55.583863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3ef4d-96b0-4fb3-aaa1-abdc3b68a9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:55.584552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3ef4d-96b0-4fb3-aaa1-abdc3b68a9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:55.584826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3ef4d-96b0-4fb3-aaa1-abdc3b68a9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:55.585092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3ef4d-96b0-4fb3-aaa1-abdc3b68a9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:55.586745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-82d3ef4d-96b0-4fb3-aaa1-abdc3b68a9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:55.702208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:55.704677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3ef4d-96b0-4fb3-aaa1-abdc3b68a9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:55.706099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3ef4d-96b0-4fb3-aaa1-abdc3b68a9e4 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:55.724954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 669/1328] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:20:57.699599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6511f7c-97b4-491a-8986-ca4e861cea39 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:57.700136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6511f7c-97b4-491a-8986-ca4e861cea39 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:57.700136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6511f7c-97b4-491a-8986-ca4e861cea39 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:57.701737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 660/1329] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:20:57.707482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b35ec7-ea87-4ba2-827c-5b25801fa0d1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:57.708035 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b35ec7-ea87-4ba2-827c-5b25801fa0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:57.708118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b35ec7-ea87-4ba2-827c-5b25801fa0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:57.794475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-62b35ec7-ea87-4ba2-827c-5b25801fa0d1 None None] MFA Rules not processed for user `6b6757539b0548b8a38441652f18bce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:20:57.795717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-62b35ec7-ea87-4ba2-827c-5b25801fa0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:57.905696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 670/1330] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:20:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:20:58.307638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6511f7c-97b4-491a-8986-ca4e861cea39 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:20:58.309092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f6511f7c-97b4-491a-8986-ca4e861cea39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:58.358844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:58.450033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6511f7c-97b4-491a-8986-ca4e861cea39 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:20:58.555051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08661a0f-f1ff-4d1f-92c2-eac99a6ad3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:20:58.556494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08661a0f-f1ff-4d1f-92c2-eac99a6ad3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:20:58.556978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08661a0f-f1ff-4d1f-92c2-eac99a6ad3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:20:58.557432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08661a0f-f1ff-4d1f-92c2-eac99a6ad3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:20:58.560546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-08661a0f-f1ff-4d1f-92c2-eac99a6ad3f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:20:58.671702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:20:58.712727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08661a0f-f1ff-4d1f-92c2-eac99a6ad3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:20:58.714245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08661a0f-f1ff-4d1f-92c2-eac99a6ad3f5 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:20:58.735276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 661/1331] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:20:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:01.201131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3aea01-7530-43f9-b2f3-b3337bdee7d4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:01.201757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3aea01-7530-43f9-b2f3-b3337bdee7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:01.201757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3aea01-7530-43f9-b2f3-b3337bdee7d4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:01.203082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 671/1332] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:01.208918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f157d9-69c7-4b7e-8a72-ced01604c9ce None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:01.209368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f157d9-69c7-4b7e-8a72-ced01604c9ce None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:01.209462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f157d9-69c7-4b7e-8a72-ced01604c9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:01.278096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-84f157d9-69c7-4b7e-8a72-ced01604c9ce None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:01.279309 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-84f157d9-69c7-4b7e-8a72-ced01604c9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:01.381548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 662/1333] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:01.668146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3aea01-7530-43f9-b2f3-b3337bdee7d4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:01.669685 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4d3aea01-7530-43f9-b2f3-b3337bdee7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:01.746273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:01.780864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3aea01-7530-43f9-b2f3-b3337bdee7d4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:01.855367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d55a-0a74-4dbd-b82a-4d6dace7a50c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:01.856001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d55a-0a74-4dbd-b82a-4d6dace7a50c service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:01.856310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d55a-0a74-4dbd-b82a-4d6dace7a50c service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:01.856580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d55a-0a74-4dbd-b82a-4d6dace7a50c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:01.858173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4945d55a-0a74-4dbd-b82a-4d6dace7a50c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:01.976117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945d55a-0a74-4dbd-b82a-4d6dace7a50c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:01.978203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945d55a-0a74-4dbd-b82a-4d6dace7a50c service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:01.989602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:01.999959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 672/1334] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:03.998792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144a3a3-7d5e-41d7-879f-e806658e5c8e None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:03.999323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144a3a3-7d5e-41d7-879f-e806658e5c8e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:03.999323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144a3a3-7d5e-41d7-879f-e806658e5c8e None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:04.000714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 663/1335] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:04.006158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e10497-f00e-4334-bd38-658416ed683e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:04.006643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e10497-f00e-4334-bd38-658416ed683e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:04.006749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e10497-f00e-4334-bd38-658416ed683e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:04.099021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-61e10497-f00e-4334-bd38-658416ed683e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:04.100397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-61e10497-f00e-4334-bd38-658416ed683e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:04.205453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 673/1336] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:04.493303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144a3a3-7d5e-41d7-879f-e806658e5c8e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:04.494914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3144a3a3-7d5e-41d7-879f-e806658e5c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:04.536798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:04.604321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144a3a3-7d5e-41d7-879f-e806658e5c8e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:04.684616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015da10-dfa8-4b25-9021-df141ffe98e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:04.685218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015da10-dfa8-4b25-9021-df141ffe98e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:04.685495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015da10-dfa8-4b25-9021-df141ffe98e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:04.685764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015da10-dfa8-4b25-9021-df141ffe98e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:04.687414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3015da10-dfa8-4b25-9021-df141ffe98e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:04.781592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:04.811424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015da10-dfa8-4b25-9021-df141ffe98e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:04.814173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015da10-dfa8-4b25-9021-df141ffe98e6 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:04.838397 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 664/1337] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:21:07.437646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29255d13-c8b7-4a5b-b421-e208c1663a94 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:07.438180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29255d13-c8b7-4a5b-b421-e208c1663a94 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:07.438180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29255d13-c8b7-4a5b-b421-e208c1663a94 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:07.439462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 674/1338] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:07.445456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c825ef-3d6e-4f1d-b1b2-3c28ec0eda3f None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:07.446468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c825ef-3d6e-4f1d-b1b2-3c28ec0eda3f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:07.446629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c825ef-3d6e-4f1d-b1b2-3c28ec0eda3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:07.521084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c0c825ef-3d6e-4f1d-b1b2-3c28ec0eda3f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:07.522466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c0c825ef-3d6e-4f1d-b1b2-3c28ec0eda3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:07.624350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 665/1339] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:07.905124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29255d13-c8b7-4a5b-b421-e208c1663a94 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:07.906662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-29255d13-c8b7-4a5b-b421-e208c1663a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:07.992155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:08.022846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29255d13-c8b7-4a5b-b421-e208c1663a94 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:08.105998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8abf1-32dc-4da9-9f46-023f700cfd81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:08.106608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8abf1-32dc-4da9-9f46-023f700cfd81 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:08.106887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8abf1-32dc-4da9-9f46-023f700cfd81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:08.107157 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8abf1-32dc-4da9-9f46-023f700cfd81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:08.108832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-91a8abf1-32dc-4da9-9f46-023f700cfd81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:08.224578 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8abf1-32dc-4da9-9f46-023f700cfd81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:08.226053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8abf1-32dc-4da9-9f46-023f700cfd81 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:08.246453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 675/1340] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:12.547104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006cbdc-31f3-43b5-9940-7b9c417888e8 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:12.547985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006cbdc-31f3-43b5-9940-7b9c417888e8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:12.547985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006cbdc-31f3-43b5-9940-7b9c417888e8 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:12.549567 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 666/1341] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:12.556513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74585e-8621-4bf7-af5c-c915b2aec90d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:12.562532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74585e-8621-4bf7-af5c-c915b2aec90d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:12.562532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74585e-8621-4bf7-af5c-c915b2aec90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:12.563203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:12.637488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ad74585e-8621-4bf7-af5c-c915b2aec90d None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:12.638744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ad74585e-8621-4bf7-af5c-c915b2aec90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:12.758076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 676/1342] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:13.044774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006cbdc-31f3-43b5-9940-7b9c417888e8 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:13.046359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7006cbdc-31f3-43b5-9940-7b9c417888e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:13.099401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:13.156695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006cbdc-31f3-43b5-9940-7b9c417888e8 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:13.240930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a455a7-2d82-4c82-9a93-c21bc69a8dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:13.241536 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a455a7-2d82-4c82-9a93-c21bc69a8dca service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:13.241816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a455a7-2d82-4c82-9a93-c21bc69a8dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:13.242088 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a455a7-2d82-4c82-9a93-c21bc69a8dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:13.244313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f3a455a7-2d82-4c82-9a93-c21bc69a8dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:13.346732 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:13.366411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a455a7-2d82-4c82-9a93-c21bc69a8dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:13.367911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a455a7-2d82-4c82-9a93-c21bc69a8dca service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:13.387471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 667/1343] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:16.409446 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5c14-bdce-45ea-9530-88ae9a8bed1b None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:16.409974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5c14-bdce-45ea-9530-88ae9a8bed1b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:16.409974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5c14-bdce-45ea-9530-88ae9a8bed1b None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:16.411328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 677/1344] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:16.416566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f4cdf-9a66-420f-bbea-484ac17b3059 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:16.417038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f4cdf-9a66-420f-bbea-484ac17b3059 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:16.417145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f4cdf-9a66-420f-bbea-484ac17b3059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:16.489198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6a0f4cdf-9a66-420f-bbea-484ac17b3059 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:16.490349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6a0f4cdf-9a66-420f-bbea-484ac17b3059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:16.619697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 668/1345] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:16.907904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a5c14-bdce-45ea-9530-88ae9a8bed1b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:16.909523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-609a5c14-bdce-45ea-9530-88ae9a8bed1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:17.005899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:17.023173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a5c14-bdce-45ea-9530-88ae9a8bed1b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:17.111712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9891d7-9fab-4700-9f46-e766f5134e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:17.112240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9891d7-9fab-4700-9f46-e766f5134e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:17.112515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9891d7-9fab-4700-9f46-e766f5134e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:17.112790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9891d7-9fab-4700-9f46-e766f5134e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:17.114451 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5a9891d7-9fab-4700-9f46-e766f5134e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:17.239310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9891d7-9fab-4700-9f46-e766f5134e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:17.241267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9891d7-9fab-4700-9f46-e766f5134e33 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:17.263988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 678/1346] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:21.640474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c7efb-e913-48da-89df-5347e5096d41 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:21.641011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c7efb-e913-48da-89df-5347e5096d41 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:21.641011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c7efb-e913-48da-89df-5347e5096d41 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:21.642383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 669/1347] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:21.648174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3775e5b-6287-4474-a68a-ac31ba7bc056 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:21.648693 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3775e5b-6287-4474-a68a-ac31ba7bc056 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:21.648693 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3775e5b-6287-4474-a68a-ac31ba7bc056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:21.665705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:21.723158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a3775e5b-6287-4474-a68a-ac31ba7bc056 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:21.724440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a3775e5b-6287-4474-a68a-ac31ba7bc056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:21.836394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 679/1348] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:22.123268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c7efb-e913-48da-89df-5347e5096d41 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:22.124895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-339c7efb-e913-48da-89df-5347e5096d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:22.189196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:22.245484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c7efb-e913-48da-89df-5347e5096d41 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:22.319018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad75603-52ee-4cd7-8cd9-62f8de8c0cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:22.319666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75603-52ee-4cd7-8cd9-62f8de8c0cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:22.319954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75603-52ee-4cd7-8cd9-62f8de8c0cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:22.320226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75603-52ee-4cd7-8cd9-62f8de8c0cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:22.322435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4ad75603-52ee-4cd7-8cd9-62f8de8c0cf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:22.428565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:22.432209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad75603-52ee-4cd7-8cd9-62f8de8c0cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:22.433646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad75603-52ee-4cd7-8cd9-62f8de8c0cf5 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:22.453145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 670/1349] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:27.890854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c613966-49fb-4957-9866-c97b8b7b7b97 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:27.891356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c613966-49fb-4957-9866-c97b8b7b7b97 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:27.891692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c613966-49fb-4957-9866-c97b8b7b7b97 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:27.893647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 680/1350] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:21:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:27.899168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e520-de23-4fc1-8399-7f89ced2a1df None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:27.899698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e520-de23-4fc1-8399-7f89ced2a1df None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:27.899998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e520-de23-4fc1-8399-7f89ced2a1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:28.022653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-3e67e520-de23-4fc1-8399-7f89ced2a1df None None] MFA Rules not processed for user `90d65372febb414d82843f9dfa56f738`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:28.025115 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3e67e520-de23-4fc1-8399-7f89ced2a1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:28.209203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 671/1351] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:21:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:31.833747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=59 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:32.852173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce344-7a2b-46d0-a7b6-9eb046534202 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.852850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce344-7a2b-46d0-a7b6-9eb046534202 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.852850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce344-7a2b-46d0-a7b6-9eb046534202 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.854851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 681/1352] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:21:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:32.860146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bf417-bf0a-4eb3-9976-ba7aa6ad8f54 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:32.860652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bf417-bf0a-4eb3-9976-ba7aa6ad8f54 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:32.860771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bf417-bf0a-4eb3-9976-ba7aa6ad8f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:32.920575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:32.944932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-2e8bf417-bf0a-4eb3-9976-ba7aa6ad8f54 None None] MFA Rules not processed for user `90d65372febb414d82843f9dfa56f738`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:32.946350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2e8bf417-bf0a-4eb3-9976-ba7aa6ad8f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:33.092645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 672/1353] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:21:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:34.543995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb9d59-3e6a-4bef-9b8a-48bf8b0f0728 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:34.544516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb9d59-3e6a-4bef-9b8a-48bf8b0f0728 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:34.544516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb9d59-3e6a-4bef-9b8a-48bf8b0f0728 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:34.545903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 682/1354] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:34.551113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0fc7f-8f39-4031-bb86-63faf0c1a875 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:34.551583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0fc7f-8f39-4031-bb86-63faf0c1a875 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:34.551855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0fc7f-8f39-4031-bb86-63faf0c1a875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:34.632403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ddb0fc7f-8f39-4031-bb86-63faf0c1a875 None None] MFA Rules not processed for user `90d65372febb414d82843f9dfa56f738`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:34.633606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ddb0fc7f-8f39-4031-bb86-63faf0c1a875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:34.677110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:34.746063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 673/1355] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:21:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:38.145931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c9efc-0501-4ec0-a70b-f85bdfc0b7ba None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:38.146651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c9efc-0501-4ec0-a70b-f85bdfc0b7ba None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:38.146651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c9efc-0501-4ec0-a70b-f85bdfc0b7ba None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:38.147979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 683/1356] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:38.154425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c732b-d1b3-435f-aee3-5526f5314d01 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:38.154919 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c732b-d1b3-435f-aee3-5526f5314d01 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:38.155024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c732b-d1b3-435f-aee3-5526f5314d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:38.248696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-128c732b-d1b3-435f-aee3-5526f5314d01 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:38.250242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-128c732b-d1b3-435f-aee3-5526f5314d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:38.415884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 674/1357] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:38.734526 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95465799-ed0c-43f0-9832-155c0852a8c3 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:38.734826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95465799-ed0c-43f0-9832-155c0852a8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:38.735087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95465799-ed0c-43f0-9832-155c0852a8c3 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:38.736649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 684/1358] 10.222.0.62 () {54 vars in 962 bytes} [Mon Sep 5 12:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:38.752840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51c4d1-f035-48b3-8463-b242aade4cd6 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:38.753130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51c4d1-f035-48b3-8463-b242aade4cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:38.753485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51c4d1-f035-48b3-8463-b242aade4cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:38.781505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:38.859366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ce51c4d1-f035-48b3-8463-b242aade4cd6 None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:38.860612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ce51c4d1-f035-48b3-8463-b242aade4cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:39.003927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 675/1359] 10.222.0.62 () {58 vars in 1059 bytes} [Mon Sep 5 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:39.009202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95465799-ed0c-43f0-9832-155c0852a8c3 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:39.010720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-95465799-ed0c-43f0-9832-155c0852a8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:39.176993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95465799-ed0c-43f0-9832-155c0852a8c3 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:39.258137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:39.305836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881d319-d2c9-4500-8ad3-734df7d011d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:39.306985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881d319-d2c9-4500-8ad3-734df7d011d1 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:39.307496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881d319-d2c9-4500-8ad3-734df7d011d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:39.308024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881d319-d2c9-4500-8ad3-734df7d011d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:39.310012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3881d319-d2c9-4500-8ad3-734df7d011d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:39.418012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881d319-d2c9-4500-8ad3-734df7d011d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:39.420036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881d319-d2c9-4500-8ad3-734df7d011d1 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:39.440637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 685/1360] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:41.609668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c61f8-d7d5-4a6f-9482-1cb38cce42e4 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.610219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c61f8-d7d5-4a6f-9482-1cb38cce42e4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.610219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c61f8-d7d5-4a6f-9482-1cb38cce42e4 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.611727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 676/1361] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:41.617005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88e8f1-d0b0-4f9d-b49b-00cf2db47813 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:41.617476 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88e8f1-d0b0-4f9d-b49b-00cf2db47813 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:41.617581 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88e8f1-d0b0-4f9d-b49b-00cf2db47813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:41.707232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4f88e8f1-d0b0-4f9d-b49b-00cf2db47813 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:41.708566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4f88e8f1-d0b0-4f9d-b49b-00cf2db47813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:41.760877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:41.830976 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 686/1362] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:42.137698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c61f8-d7d5-4a6f-9482-1cb38cce42e4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:42.139225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c92c61f8-d7d5-4a6f-9482-1cb38cce42e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.218293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:42.320138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c61f8-d7d5-4a6f-9482-1cb38cce42e4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:42.457105 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d253a-3f1e-4a5b-846d-5e1b5018f5b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:42.457933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d253a-3f1e-4a5b-846d-5e1b5018f5b2 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:42.458215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d253a-3f1e-4a5b-846d-5e1b5018f5b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:42.458638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d253a-3f1e-4a5b-846d-5e1b5018f5b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:42.460887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-085d253a-3f1e-4a5b-846d-5e1b5018f5b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:42.495947 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:42.575130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d253a-3f1e-4a5b-846d-5e1b5018f5b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:42.576887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d253a-3f1e-4a5b-846d-5e1b5018f5b2 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:42.599474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 677/1363] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:45.102991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a25656-7159-48e2-9f6b-2bbdab9d8e3e None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:45.103601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a25656-7159-48e2-9f6b-2bbdab9d8e3e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:45.103601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a25656-7159-48e2-9f6b-2bbdab9d8e3e None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:45.104900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 687/1364] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:45.110332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531d37b-19eb-4c5e-84ae-4e9eccdd982e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:45.110743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531d37b-19eb-4c5e-84ae-4e9eccdd982e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:45.110874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531d37b-19eb-4c5e-84ae-4e9eccdd982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:45.184805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8531d37b-19eb-4c5e-84ae-4e9eccdd982e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:45.186024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8531d37b-19eb-4c5e-84ae-4e9eccdd982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:45.298767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:45.302947 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 678/1365] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:45.606207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebeb534-bc58-43dd-8223-8b88cae24f25 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:45.606468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebeb534-bc58-43dd-8223-8b88cae24f25 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:45.606728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebeb534-bc58-43dd-8223-8b88cae24f25 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:45.608655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 688/1366] 10.222.0.62 () {54 vars in 962 bytes} [Mon Sep 5 12:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:21:45.617600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9345f-0115-4844-abf0-c03d3bd83a83 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:45.625433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9345f-0115-4844-abf0-c03d3bd83a83 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:45.625433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9345f-0115-4844-abf0-c03d3bd83a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:45.709737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e4c9345f-0115-4844-abf0-c03d3bd83a83 None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:45.710888 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e4c9345f-0115-4844-abf0-c03d3bd83a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:45.868933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 679/1367] 10.222.0.62 () {58 vars in 1059 bytes} [Mon Sep 5 12:21:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:45.876963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebeb534-bc58-43dd-8223-8b88cae24f25 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:45.878895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bebeb534-bc58-43dd-8223-8b88cae24f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:46.056617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebeb534-bc58-43dd-8223-8b88cae24f25 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:46.084416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:46.190389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e502ae-27a0-4344-8373-298f713836b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:46.191047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e502ae-27a0-4344-8373-298f713836b4 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:46.191443 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e502ae-27a0-4344-8373-298f713836b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:46.191839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e502ae-27a0-4344-8373-298f713836b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:46.193609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-37e502ae-27a0-4344-8373-298f713836b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:46.291455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e502ae-27a0-4344-8373-298f713836b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:46.293257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e502ae-27a0-4344-8373-298f713836b4 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:46.319432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 689/1368] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:46.612323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3e22b-dc37-4163-b482-d7ef109a535f None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:46.612856 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3e22b-dc37-4163-b482-d7ef109a535f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:46.612856 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3e22b-dc37-4163-b482-d7ef109a535f None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:46.614438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 680/1369] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:46.620372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b0099-1ae7-478a-a7fe-1d31873cee86 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:46.620758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b0099-1ae7-478a-a7fe-1d31873cee86 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:46.620929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b0099-1ae7-478a-a7fe-1d31873cee86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:46.655449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:46.696026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7f6b0099-1ae7-478a-a7fe-1d31873cee86 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:46.697884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7f6b0099-1ae7-478a-a7fe-1d31873cee86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:46.807591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 690/1370] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:47.376481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a252e1-c810-4f1e-b3e6-a2d7fc8b797e None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:47.376815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a252e1-c810-4f1e-b3e6-a2d7fc8b797e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:47.377029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a252e1-c810-4f1e-b3e6-a2d7fc8b797e None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:47.378829 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 681/1371] 10.222.0.62 () {54 vars in 964 bytes} [Mon Sep 5 12:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:47.385360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746059b0-891e-46a2-8016-d0ad05109ceb None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:47.385868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746059b0-891e-46a2-8016-d0ad05109ceb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:47.386054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746059b0-891e-46a2-8016-d0ad05109ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:47.472093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-746059b0-891e-46a2-8016-d0ad05109ceb None None] MFA Rules not processed for user `8f33f1f3d1ca4436887c2346bd9fa149`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:47.473383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-746059b0-891e-46a2-8016-d0ad05109ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:47.529530 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:47.562978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 691/1372] 10.222.0.62 () {58 vars in 1061 bytes} [Mon Sep 5 12:21:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:47.570905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a252e1-c810-4f1e-b3e6-a2d7fc8b797e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:47.572541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-47a252e1-c810-4f1e-b3e6-a2d7fc8b797e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:47.631652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:47.690194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a252e1-c810-4f1e-b3e6-a2d7fc8b797e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:47.771891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56ed7d-7a62-48b8-bf0b-44f4f9d47096 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:47.772512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56ed7d-7a62-48b8-bf0b-44f4f9d47096 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:47.772794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56ed7d-7a62-48b8-bf0b-44f4f9d47096 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:47.773069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56ed7d-7a62-48b8-bf0b-44f4f9d47096 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:47.774734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7a56ed7d-7a62-48b8-bf0b-44f4f9d47096 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:47.889696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:47.904661 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56ed7d-7a62-48b8-bf0b-44f4f9d47096 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:47.906230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56ed7d-7a62-48b8-bf0b-44f4f9d47096 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:47.938473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 682/1373] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:48.729203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294f2a5-0a62-485c-bec1-583123c495af None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:48.729793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294f2a5-0a62-485c-bec1-583123c495af None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:48.729793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294f2a5-0a62-485c-bec1-583123c495af None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:48.731250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 692/1374] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:48.736707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2882e56-763d-4a35-8b88-e2cc2620dc17 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:48.737232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2882e56-763d-4a35-8b88-e2cc2620dc17 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:48.737335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2882e56-763d-4a35-8b88-e2cc2620dc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:48.816553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-f2882e56-763d-4a35-8b88-e2cc2620dc17 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:48.817868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f2882e56-763d-4a35-8b88-e2cc2620dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:48.923127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 683/1375] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:49.217442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294f2a5-0a62-485c-bec1-583123c495af None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:49.219423 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0294f2a5-0a62-485c-bec1-583123c495af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:49.378833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294f2a5-0a62-485c-bec1-583123c495af None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:49.437193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:49.505838 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66523efe-2c16-46f0-bb6a-15f15dd19caa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:49.506585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523efe-2c16-46f0-bb6a-15f15dd19caa service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:49.506709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523efe-2c16-46f0-bb6a-15f15dd19caa service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:49.506982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523efe-2c16-46f0-bb6a-15f15dd19caa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:49.508699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-66523efe-2c16-46f0-bb6a-15f15dd19caa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:49.609407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66523efe-2c16-46f0-bb6a-15f15dd19caa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:49.611035 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66523efe-2c16-46f0-bb6a-15f15dd19caa service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:49.630631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 693/1376] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:51.839817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabf7bf-640f-40ce-a9ba-aac9e59364db None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:51.840328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabf7bf-640f-40ce-a9ba-aac9e59364db None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:51.840328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabf7bf-640f-40ce-a9ba-aac9e59364db None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:51.846316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 684/1377] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:51.848287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8e7be-4486-46dd-878c-4854bd8b3dd2 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:51.848758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8e7be-4486-46dd-878c-4854bd8b3dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:51.848870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8e7be-4486-46dd-878c-4854bd8b3dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:51.942215 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-afd8e7be-4486-46dd-878c-4854bd8b3dd2 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:51.943462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-afd8e7be-4486-46dd-878c-4854bd8b3dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:51.952691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:52.071572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 694/1378] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:21:52.364243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabf7bf-640f-40ce-a9ba-aac9e59364db None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:52.365922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-adabf7bf-640f-40ce-a9ba-aac9e59364db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:52.432649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:52.531396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabf7bf-640f-40ce-a9ba-aac9e59364db None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:52.679148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c24b03-9a76-4f3c-a187-f68fb6e998f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:52.679938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c24b03-9a76-4f3c-a187-f68fb6e998f0 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:52.680619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c24b03-9a76-4f3c-a187-f68fb6e998f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:52.680898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c24b03-9a76-4f3c-a187-f68fb6e998f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:52.683771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-96c24b03-9a76-4f3c-a187-f68fb6e998f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:52.710516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:52.800530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c24b03-9a76-4f3c-a187-f68fb6e998f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:52.802296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c24b03-9a76-4f3c-a187-f68fb6e998f0 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:52.822711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 685/1379] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:55.176928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7767e-665b-44c4-bbcc-0a8653c61212 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:55.177456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7767e-665b-44c4-bbcc-0a8653c61212 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:55.177456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7767e-665b-44c4-bbcc-0a8653c61212 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:55.179086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 695/1380] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:55.184767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db224f-84b9-468b-9f96-2c9592b325b3 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:55.185334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db224f-84b9-468b-9f96-2c9592b325b3 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:55.185334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db224f-84b9-468b-9f96-2c9592b325b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:55.258386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d0db224f-84b9-468b-9f96-2c9592b325b3 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:55.260294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d0db224f-84b9-468b-9f96-2c9592b325b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:55.363745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:55.371073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 686/1381] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:55.664867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7767e-665b-44c4-bbcc-0a8653c61212 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:55.666476 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e3b7767e-665b-44c4-bbcc-0a8653c61212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:55.836495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:55.856350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7767e-665b-44c4-bbcc-0a8653c61212 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:55.999663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdef792-87da-4d87-bb94-7ac7d3e70d7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:56.000289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdef792-87da-4d87-bb94-7ac7d3e70d7c service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:56.002098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdef792-87da-4d87-bb94-7ac7d3e70d7c service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:56.002098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdef792-87da-4d87-bb94-7ac7d3e70d7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:56.002652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2cdef792-87da-4d87-bb94-7ac7d3e70d7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:56.123120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdef792-87da-4d87-bb94-7ac7d3e70d7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:56.125263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdef792-87da-4d87-bb94-7ac7d3e70d7c service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:56.139011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:56.149803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 696/1382] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:21:58.635727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb37f2-1ea0-4e34-a6d2-171cb368c142 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:58.636443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb37f2-1ea0-4e34-a6d2-171cb368c142 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:58.636443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb37f2-1ea0-4e34-a6d2-171cb368c142 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:58.637608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 687/1383] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:21:58.643604 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceba297-93ef-44e6-8ceb-9cf8fe3ada95 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:58.644031 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceba297-93ef-44e6-8ceb-9cf8fe3ada95 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:58.644194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceba297-93ef-44e6-8ceb-9cf8fe3ada95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:58.719256 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3ceba297-93ef-44e6-8ceb-9cf8fe3ada95 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:21:58.720796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3ceba297-93ef-44e6-8ceb-9cf8fe3ada95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:58.846183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 697/1384] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:21:59.165143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb37f2-1ea0-4e34-a6d2-171cb368c142 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:21:59.167231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cddb37f2-1ea0-4e34-a6d2-171cb368c142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:59.325293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb37f2-1ea0-4e34-a6d2-171cb368c142 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:21:59.407697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:21:59.460272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2f00b-7446-43ca-9c15-ccc4b99001ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:21:59.461054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2f00b-7446-43ca-9c15-ccc4b99001ce service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:21:59.461351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2f00b-7446-43ca-9c15-ccc4b99001ce service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:21:59.461797 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2f00b-7446-43ca-9c15-ccc4b99001ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:21:59.464141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-01b2f00b-7446-43ca-9c15-ccc4b99001ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:21:59.585063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2f00b-7446-43ca-9c15-ccc4b99001ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:21:59.586737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2f00b-7446-43ca-9c15-ccc4b99001ce service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:21:59.608107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 688/1385] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:21:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:01.667771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc5373-b6bb-457d-bb43-b813a0c4376a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.668405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc5373-b6bb-457d-bb43-b813a0c4376a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.668853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc5373-b6bb-457d-bb43-b813a0c4376a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.672003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 698/1386] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:22:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:22:01.681703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464c2b6-d1c1-4f5c-9c30-04a629d8e223 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:01.682625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464c2b6-d1c1-4f5c-9c30-04a629d8e223 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:01.682625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464c2b6-d1c1-4f5c-9c30-04a629d8e223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:01.756219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1464c2b6-d1c1-4f5c-9c30-04a629d8e223 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:01.757456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1464c2b6-d1c1-4f5c-9c30-04a629d8e223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:01.802594 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:01.882523 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 689/1387] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:22:02.176634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc5373-b6bb-457d-bb43-b813a0c4376a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:02.178164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-87cc5373-b6bb-457d-bb43-b813a0c4376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:02.258357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:02.331993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc5373-b6bb-457d-bb43-b813a0c4376a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:02.462222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68774e-6b6e-41e7-9970-fa562dad16de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:02.462841 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68774e-6b6e-41e7-9970-fa562dad16de service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:02.463291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68774e-6b6e-41e7-9970-fa562dad16de service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:02.463603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68774e-6b6e-41e7-9970-fa562dad16de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:02.465320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2c68774e-6b6e-41e7-9970-fa562dad16de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:02.507272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:02.562762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68774e-6b6e-41e7-9970-fa562dad16de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:02.564413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68774e-6b6e-41e7-9970-fa562dad16de service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:02.584334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 699/1388] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:03.641724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aec2f-94cd-4fc0-b002-0dc518032bfa None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:03.642330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aec2f-94cd-4fc0-b002-0dc518032bfa None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:03.642330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aec2f-94cd-4fc0-b002-0dc518032bfa None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:03.644076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 690/1389] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:03.651868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a8ff8-913a-4235-959e-66db073b8d75 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:03.651868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a8ff8-913a-4235-959e-66db073b8d75 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:03.651868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a8ff8-913a-4235-959e-66db073b8d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:03.726431 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0a5a8ff8-913a-4235-959e-66db073b8d75 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:03.727673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0a5a8ff8-913a-4235-959e-66db073b8d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:03.849441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 700/1390] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:04.162055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd341-dad0-4411-a061-7e07c0e40cdd None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:04.162335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd341-dad0-4411-a061-7e07c0e40cdd None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:04.162610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd341-dad0-4411-a061-7e07c0e40cdd None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:04.164160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 691/1391] 10.222.0.62 () {54 vars in 961 bytes} [Mon Sep 5 12:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:04.169880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167e9b4-0918-4024-a427-56dc44aeeefd None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:04.170163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167e9b4-0918-4024-a427-56dc44aeeefd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:04.170436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167e9b4-0918-4024-a427-56dc44aeeefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:04.187304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:04.254205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1167e9b4-0918-4024-a427-56dc44aeeefd None None] MFA Rules not processed for user `237bbeebe2a949aba444353c1d2ee04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:04.255333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1167e9b4-0918-4024-a427-56dc44aeeefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:04.338876 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 701/1392] 10.222.0.62 () {58 vars in 1058 bytes} [Mon Sep 5 12:22:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:04.346688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdd341-dad0-4411-a061-7e07c0e40cdd None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:04.348436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c8cdd341-dad0-4411-a061-7e07c0e40cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:04.458023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdd341-dad0-4411-a061-7e07c0e40cdd None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:04.526827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:04.549089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c5502-c4f9-44e7-b0e2-849ee7a2b40f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '237bbeebe2a949aba444353c1d2ee04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:04.549711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c5502-c4f9-44e7-b0e2-849ee7a2b40f service glance] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:04.550036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c5502-c4f9-44e7-b0e2-849ee7a2b40f service glance] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:04.550311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c5502-c4f9-44e7-b0e2-849ee7a2b40f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:04.552143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-856c5502-c4f9-44e7-b0e2-849ee7a2b40f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:04.656437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0b56-b68b-402c-a299-220fd500d1b7 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:04.657000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0b56-b68b-402c-a299-220fd500d1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:04.657000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0b56-b68b-402c-a299-220fd500d1b7 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:04.658362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 702/1393] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:22:04.663498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d844674-ade8-4838-b51d-2cb2d9e73775 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:04.663949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d844674-ade8-4838-b51d-2cb2d9e73775 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:04.667087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d844674-ade8-4838-b51d-2cb2d9e73775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:04.671107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c5502-c4f9-44e7-b0e2-849ee7a2b40f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:04.673237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c5502-c4f9-44e7-b0e2-849ee7a2b40f service glance] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:04.694271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 692/1394] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:04.742700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9d844674-ade8-4838-b51d-2cb2d9e73775 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:04.744213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9d844674-ade8-4838-b51d-2cb2d9e73775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:04.800242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:04.843469 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 703/1395] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:22:05.131923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c5502-c4f9-44e7-b0e2-849ee7a2b40f service glance] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:05.133199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-856c5502-c4f9-44e7-b0e2-849ee7a2b40f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:05.245005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:05.253505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c5502-c4f9-44e7-b0e2-849ee7a2b40f service glance] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:05.380675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a8ca-809b-4e22-986f-bc02e28d75c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:05.381323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a8ca-809b-4e22-986f-bc02e28d75c7 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:05.381601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a8ca-809b-4e22-986f-bc02e28d75c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:05.381872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a8ca-809b-4e22-986f-bc02e28d75c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:05.383668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2c59a8ca-809b-4e22-986f-bc02e28d75c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:05.494977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59a8ca-809b-4e22-986f-bc02e28d75c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:05.496713 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59a8ca-809b-4e22-986f-bc02e28d75c7 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:05.513749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:05.516925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 693/1396] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:05.676405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b710f1-3453-410e-a661-55086394b709 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:05.676871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b710f1-3453-410e-a661-55086394b709 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:05.677162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b710f1-3453-410e-a661-55086394b709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:05.750275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e2b710f1-3453-410e-a661-55086394b709 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:05.751519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e2b710f1-3453-410e-a661-55086394b709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:05.857106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 704/1397] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:05.906565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f194c4-dfb5-4f46-a402-8722acb16799 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:05.907165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f194c4-dfb5-4f46-a402-8722acb16799 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:05.907165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f194c4-dfb5-4f46-a402-8722acb16799 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:05.908630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 694/1398] 10.222.0.62 () {54 vars in 943 bytes} [Mon Sep 5 12:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:05.916671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6a496-4746-4a24-aa08-3b452cea78ef None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:05.917081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6a496-4746-4a24-aa08-3b452cea78ef None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:05.917521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6a496-4746-4a24-aa08-3b452cea78ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:05.987407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:06.014003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-65b6a496-4746-4a24-aa08-3b452cea78ef None None] MFA Rules not processed for user `b48aa41ff66b42329926bc42379d9800`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:06.015259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-65b6a496-4746-4a24-aa08-3b452cea78ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:06.138098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 705/1399] 10.222.0.62 () {58 vars in 1040 bytes} [Mon Sep 5 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:06.147940 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f194c4-dfb5-4f46-a402-8722acb16799 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:06.149606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-80f194c4-dfb5-4f46-a402-8722acb16799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:06.298649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f194c4-dfb5-4f46-a402-8722acb16799 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:06.398596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:06.420482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0ff5f-81e4-409f-9795-66e5f78e99c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:06.421129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0ff5f-81e4-409f-9795-66e5f78e99c1 service swift] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:06.421414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0ff5f-81e4-409f-9795-66e5f78e99c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:06.421686 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0ff5f-81e4-409f-9795-66e5f78e99c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:06.423376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8df0ff5f-81e4-409f-9795-66e5f78e99c1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:06.515185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0ff5f-81e4-409f-9795-66e5f78e99c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:06.516896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0ff5f-81e4-409f-9795-66e5f78e99c1 service swift] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:06.527148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 695/1400] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:22:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:22:07.702484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a9c0d-1c99-4629-8dc1-a3d32c908c12 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:07.702484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a9c0d-1c99-4629-8dc1-a3d32c908c12 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:07.702484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a9c0d-1c99-4629-8dc1-a3d32c908c12 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:07.704999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 706/1401] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:22:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:07.711415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec635b34-05a7-445a-8302-500743738f30 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:07.712138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec635b34-05a7-445a-8302-500743738f30 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:07.712138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec635b34-05a7-445a-8302-500743738f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:07.796736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ec635b34-05a7-445a-8302-500743738f30 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:07.799965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ec635b34-05a7-445a-8302-500743738f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:07.883394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:07.922033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 696/1402] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:08.246992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a9c0d-1c99-4629-8dc1-a3d32c908c12 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:08.248668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4b0a9c0d-1c99-4629-8dc1-a3d32c908c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:08.377420 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:08.432232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a9c0d-1c99-4629-8dc1-a3d32c908c12 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:08.571353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ffa96-3840-491e-8360-e869a520998d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:08.572010 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ffa96-3840-491e-8360-e869a520998d service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:08.572936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ffa96-3840-491e-8360-e869a520998d service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:08.573214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ffa96-3840-491e-8360-e869a520998d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:08.574843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bf2ffa96-3840-491e-8360-e869a520998d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:08.660150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:08.684475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ffa96-3840-491e-8360-e869a520998d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:08.690660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ffa96-3840-491e-8360-e869a520998d service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:08.708028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 707/1403] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:10.914167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74f887-43f8-476d-bc93-481bcd276e83 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:10.914736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74f887-43f8-476d-bc93-481bcd276e83 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:10.914736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74f887-43f8-476d-bc93-481bcd276e83 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:10.916866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 697/1404] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:10.922483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700057c-498c-401e-8cf5-df4026fd236b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:10.922997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700057c-498c-401e-8cf5-df4026fd236b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:10.922997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700057c-498c-401e-8cf5-df4026fd236b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:11.001535 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8700057c-498c-401e-8cf5-df4026fd236b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:11.003477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8700057c-498c-401e-8cf5-df4026fd236b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:11.135475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 708/1405] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:11.432529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74f887-43f8-476d-bc93-481bcd276e83 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:11.434051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5d74f887-43f8-476d-bc93-481bcd276e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:11.608347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74f887-43f8-476d-bc93-481bcd276e83 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:11.676988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:11.749385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fad27-3022-44a8-9317-e1d100dc2c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:11.750122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fad27-3022-44a8-9317-e1d100dc2c0b service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:11.750402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fad27-3022-44a8-9317-e1d100dc2c0b service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:11.750683 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fad27-3022-44a8-9317-e1d100dc2c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:11.752414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-411fad27-3022-44a8-9317-e1d100dc2c0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:11.872182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fad27-3022-44a8-9317-e1d100dc2c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:11.873817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fad27-3022-44a8-9317-e1d100dc2c0b service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:11.900037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 698/1406] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:14.146852 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10fb4b-1f2d-4379-a32e-b32506aedbf8 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:14.147462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10fb4b-1f2d-4379-a32e-b32506aedbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:14.147462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10fb4b-1f2d-4379-a32e-b32506aedbf8 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:14.149110 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 709/1407] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:14.154894 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47d402-4127-43a4-9192-808fdc0ec13e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:14.155342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47d402-4127-43a4-9192-808fdc0ec13e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:14.155432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47d402-4127-43a4-9192-808fdc0ec13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:14.242278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-0b47d402-4127-43a4-9192-808fdc0ec13e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:14.244116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0b47d402-4127-43a4-9192-808fdc0ec13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:14.261931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:14.365053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 699/1408] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:14.657187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10fb4b-1f2d-4379-a32e-b32506aedbf8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:14.658718 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4c10fb4b-1f2d-4379-a32e-b32506aedbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:14.721059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:14.838954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10fb4b-1f2d-4379-a32e-b32506aedbf8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:14.983697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc28b61-038b-4587-82c3-06563229ac20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:14.984232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc28b61-038b-4587-82c3-06563229ac20 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:14.984567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc28b61-038b-4587-82c3-06563229ac20 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:14.984843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc28b61-038b-4587-82c3-06563229ac20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:14.986605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1dc28b61-038b-4587-82c3-06563229ac20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:14.997544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:15.105934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc28b61-038b-4587-82c3-06563229ac20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:15.105934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc28b61-038b-4587-82c3-06563229ac20 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:15.124023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 710/1409] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:24.363086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=37 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:25.121640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=41 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:30.394606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988fb9-4f78-4b77-8bcc-7342aa6c12b7 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:30.395130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988fb9-4f78-4b77-8bcc-7342aa6c12b7 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:30.395130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988fb9-4f78-4b77-8bcc-7342aa6c12b7 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:30.396482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 700/1410] 10.222.0.62 () {54 vars in 918 bytes} [Mon Sep 5 12:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:30.402362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661170b-af39-400b-b180-d37f44039375 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:30.402935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661170b-af39-400b-b180-d37f44039375 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:30.403071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661170b-af39-400b-b180-d37f44039375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:30.485470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3661170b-af39-400b-b180-d37f44039375 None None] MFA Rules not processed for user `90d65372febb414d82843f9dfa56f738`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:30.487211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3661170b-af39-400b-b180-d37f44039375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:30.595886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 711/1411] 10.222.0.62 () {58 vars in 1015 bytes} [Mon Sep 5 12:22:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:30.610967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5b03d-89c0-4a75-9463-d01f475daef6 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:30.611284 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5b03d-89c0-4a75-9463-d01f475daef6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:30.611618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5b03d-89c0-4a75-9463-d01f475daef6 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:30.612994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 701/1412] 10.222.0.62 () {54 vars in 918 bytes} [Mon Sep 5 12:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:22:30.617849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1edfeb-6287-46b4-9b30-adaa3cbc60f9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:30.618134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1edfeb-6287-46b4-9b30-adaa3cbc60f9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:30.618414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1edfeb-6287-46b4-9b30-adaa3cbc60f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:30.693218 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:30.696354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2e1edfeb-6287-46b4-9b30-adaa3cbc60f9 None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:30.697589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2e1edfeb-6287-46b4-9b30-adaa3cbc60f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:30.820054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 712/1413] 10.222.0.62 () {58 vars in 1015 bytes} [Mon Sep 5 12:22:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:33.221810 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2afc51-8934-4904-9131-d66bb3b26ece None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:33.222391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2afc51-8934-4904-9131-d66bb3b26ece None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:33.222391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2afc51-8934-4904-9131-d66bb3b26ece None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:33.223738 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:33.230315 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380c5cf-aaf0-4c06-a1ef-f1771f2d50bf None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:33.230751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380c5cf-aaf0-4c06-a1ef-f1771f2d50bf None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:33.230855 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380c5cf-aaf0-4c06-a1ef-f1771f2d50bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:33.300141 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5380c5cf-aaf0-4c06-a1ef-f1771f2d50bf None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:33.301406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5380c5cf-aaf0-4c06-a1ef-f1771f2d50bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:33.359409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:33.402584 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:33.696612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2afc51-8934-4904-9131-d66bb3b26ece None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:33.698653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7f2afc51-8934-4904-9131-d66bb3b26ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:33.863506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2afc51-8934-4904-9131-d66bb3b26ece None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:33.938975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:33.982850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714bae1-a94e-48ae-bc8e-24e2f4e5c285 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:33.983463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bae1-a94e-48ae-bc8e-24e2f4e5c285 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:33.983776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bae1-a94e-48ae-bc8e-24e2f4e5c285 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:33.984050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bae1-a94e-48ae-bc8e-24e2f4e5c285 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:33.985662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3714bae1-a94e-48ae-bc8e-24e2f4e5c285 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:34.080355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714bae1-a94e-48ae-bc8e-24e2f4e5c285 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:34.082025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714bae1-a94e-48ae-bc8e-24e2f4e5c285 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:34.100674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 703/1416] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:35.017484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b0460-55d5-4ebb-b29a-e43c85b09dc6 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:35.018109 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b0460-55d5-4ebb-b29a-e43c85b09dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:35.018109 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b0460-55d5-4ebb-b29a-e43c85b09dc6 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:35.019350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:35.024514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ee83e-8a3c-4695-8ce3-f59405d7b430 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:35.025070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ee83e-8a3c-4695-8ce3-f59405d7b430 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:35.025184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ee83e-8a3c-4695-8ce3-f59405d7b430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:35.120191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-591ee83e-8a3c-4695-8ce3-f59405d7b430 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:35.121514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-591ee83e-8a3c-4695-8ce3-f59405d7b430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:35.156393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:35.240427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:35.395200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b0460-55d5-4ebb-b29a-e43c85b09dc6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:35.396812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4a1b0460-55d5-4ebb-b29a-e43c85b09dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:35.547619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b0460-55d5-4ebb-b29a-e43c85b09dc6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:35.590029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:35.664808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbc6c6-fc6c-4d48-9a7c-bbc5687869c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:35.665421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbc6c6-fc6c-4d48-9a7c-bbc5687869c2 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:35.665700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbc6c6-fc6c-4d48-9a7c-bbc5687869c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:35.665970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbc6c6-fc6c-4d48-9a7c-bbc5687869c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:35.667675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c7fbc6c6-fc6c-4d48-9a7c-bbc5687869c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:35.762590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbc6c6-fc6c-4d48-9a7c-bbc5687869c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:35.764368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbc6c6-fc6c-4d48-9a7c-bbc5687869c2 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:35.783088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 715/1419] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:36.694732 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e76fc5-c4ce-4aa8-9ea3-541b756c0998 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:36.695303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e76fc5-c4ce-4aa8-9ea3-541b756c0998 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:36.695303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e76fc5-c4ce-4aa8-9ea3-541b756c0998 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:36.696707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:36.701973 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62171ce8-f5ce-4768-9a17-d7d12ede489b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:36.702434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62171ce8-f5ce-4768-9a17-d7d12ede489b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:36.702560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62171ce8-f5ce-4768-9a17-d7d12ede489b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:36.760040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:36.783175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-62171ce8-f5ce-4768-9a17-d7d12ede489b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:36.784373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-62171ce8-f5ce-4768-9a17-d7d12ede489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:36.886858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:37.039846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e76fc5-c4ce-4aa8-9ea3-541b756c0998 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:37.041359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-89e76fc5-c4ce-4aa8-9ea3-541b756c0998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:37.194735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e76fc5-c4ce-4aa8-9ea3-541b756c0998 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:37.208754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:37.316758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af1e47-26b0-480f-83bd-2f4b5d4fd6b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:37.317296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1e47-26b0-480f-83bd-2f4b5d4fd6b9 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:37.317575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1e47-26b0-480f-83bd-2f4b5d4fd6b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:37.317846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1e47-26b0-480f-83bd-2f4b5d4fd6b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:37.319488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e5af1e47-26b0-480f-83bd-2f4b5d4fd6b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:37.416770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af1e47-26b0-480f-83bd-2f4b5d4fd6b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:37.418421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af1e47-26b0-480f-83bd-2f4b5d4fd6b9 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:37.438001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 706/1422] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:38.346271 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c1cae-4cf0-40fb-9077-90b5d0efde82 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:38.347084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c1cae-4cf0-40fb-9077-90b5d0efde82 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:38.347084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c1cae-4cf0-40fb-9077-90b5d0efde82 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:38.348456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:38.353833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7e959-d3e0-48d2-b151-c0626b8da55e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:38.354822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7e959-d3e0-48d2-b151-c0626b8da55e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:38.354917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7e959-d3e0-48d2-b151-c0626b8da55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:38.376471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:38.437877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-25b7e959-d3e0-48d2-b151-c0626b8da55e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:38.439114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-25b7e959-d3e0-48d2-b151-c0626b8da55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:38.542112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:38.695308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c1cae-4cf0-40fb-9077-90b5d0efde82 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:38.696863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-119c1cae-4cf0-40fb-9077-90b5d0efde82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:38.820036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:38.861953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c1cae-4cf0-40fb-9077-90b5d0efde82 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:38.986704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc88eec-28b4-4b0d-ba49-f53599a4d2f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:38.987223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc88eec-28b4-4b0d-ba49-f53599a4d2f0 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:38.987503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc88eec-28b4-4b0d-ba49-f53599a4d2f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:38.987811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc88eec-28b4-4b0d-ba49-f53599a4d2f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:38.989439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fdc88eec-28b4-4b0d-ba49-f53599a4d2f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:39.071679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:39.086065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc88eec-28b4-4b0d-ba49-f53599a4d2f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:39.087704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc88eec-28b4-4b0d-ba49-f53599a4d2f0 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:39.106870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 718/1425] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:22:39.990113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35c73d-0c41-4c4a-96d9-a79e62744e22 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:39.990668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35c73d-0c41-4c4a-96d9-a79e62744e22 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:39.990668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35c73d-0c41-4c4a-96d9-a79e62744e22 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:39.992053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:39.997358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc7d0c-106b-418c-b26f-4383433d4849 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:39.997847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc7d0c-106b-418c-b26f-4383433d4849 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:39.997945 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc7d0c-106b-418c-b26f-4383433d4849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:40.078024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-cbdc7d0c-106b-418c-b26f-4383433d4849 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:40.079238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cbdc7d0c-106b-418c-b26f-4383433d4849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:40.186667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:40.342851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35c73d-0c41-4c4a-96d9-a79e62744e22 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:40.344413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4c35c73d-0c41-4c4a-96d9-a79e62744e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:40.431793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:40.499492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35c73d-0c41-4c4a-96d9-a79e62744e22 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:40.623029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10e4a9-da25-4f7b-ac20-9e658f15e7ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:40.623581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10e4a9-da25-4f7b-ac20-9e658f15e7ca service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:40.623865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10e4a9-da25-4f7b-ac20-9e658f15e7ca service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:40.624143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10e4a9-da25-4f7b-ac20-9e658f15e7ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:40.625751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-db10e4a9-da25-4f7b-ac20-9e658f15e7ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:40.679138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:40.727450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10e4a9-da25-4f7b-ac20-9e658f15e7ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:40.729106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10e4a9-da25-4f7b-ac20-9e658f15e7ca service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:40.748759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 709/1428] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:41.639768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e67b4-0f13-4d8e-a345-0fdb3f109430 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:41.640377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e67b4-0f13-4d8e-a345-0fdb3f109430 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:41.640377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e67b4-0f13-4d8e-a345-0fdb3f109430 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:41.641637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:41.646852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd03f0-3921-458d-9889-c4fdef33bec5 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:41.647323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd03f0-3921-458d-9889-c4fdef33bec5 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:41.647435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd03f0-3921-458d-9889-c4fdef33bec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:41.728781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e0dd03f0-3921-458d-9889-c4fdef33bec5 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:41.730027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e0dd03f0-3921-458d-9889-c4fdef33bec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:41.830570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:41.992167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e67b4-0f13-4d8e-a345-0fdb3f109430 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:41.993469 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-648e67b4-0f13-4d8e-a345-0fdb3f109430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:42.059613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:42.159942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e67b4-0f13-4d8e-a345-0fdb3f109430 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:42.276565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3b0b0-837b-4091-ac35-75d654ac6d92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:42.277190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3b0b0-837b-4091-ac35-75d654ac6d92 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:42.277467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3b0b0-837b-4091-ac35-75d654ac6d92 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:42.277737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3b0b0-837b-4091-ac35-75d654ac6d92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:42.279438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a9a3b0b0-837b-4091-ac35-75d654ac6d92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:42.287247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:42.377426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3b0b0-837b-4091-ac35-75d654ac6d92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:42.379084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3b0b0-837b-4091-ac35-75d654ac6d92 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:42.398683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 721/1431] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:43.295951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e16dd5-bbfd-4202-ba95-1fabe8d28312 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:43.296853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e16dd5-bbfd-4202-ba95-1fabe8d28312 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:43.296853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e16dd5-bbfd-4202-ba95-1fabe8d28312 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:43.298419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:43.303704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dae2f-9764-4c9f-93ab-f9fc2b1dca68 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:43.304166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dae2f-9764-4c9f-93ab-f9fc2b1dca68 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:43.304166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dae2f-9764-4c9f-93ab-f9fc2b1dca68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:43.386946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2a6dae2f-9764-4c9f-93ab-f9fc2b1dca68 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:43.388212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2a6dae2f-9764-4c9f-93ab-f9fc2b1dca68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:43.465732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:43.496530 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:43.649702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e16dd5-bbfd-4202-ba95-1fabe8d28312 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:43.651338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f0e16dd5-bbfd-4202-ba95-1fabe8d28312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:43.670050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:43.811131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e16dd5-bbfd-4202-ba95-1fabe8d28312 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:43.903846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:43.931265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611c0c0-ee96-4549-b686-7642ffc8f1c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:43.931979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611c0c0-ee96-4549-b686-7642ffc8f1c4 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:43.932255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611c0c0-ee96-4549-b686-7642ffc8f1c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:43.933410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611c0c0-ee96-4549-b686-7642ffc8f1c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:43.935144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b611c0c0-ee96-4549-b686-7642ffc8f1c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:44.050459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611c0c0-ee96-4549-b686-7642ffc8f1c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:44.052166 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611c0c0-ee96-4549-b686-7642ffc8f1c4 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:44.072848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 712/1434] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:44.955575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97529b6-9367-4563-8700-421c4b4acd5a None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:44.956107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97529b6-9367-4563-8700-421c4b4acd5a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:44.956107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97529b6-9367-4563-8700-421c4b4acd5a None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:44.957501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:44.963132 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689e155-cda7-4498-8e93-d174ff99fcfe None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:44.963751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689e155-cda7-4498-8e93-d174ff99fcfe None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:44.963751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689e155-cda7-4498-8e93-d174ff99fcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:45.049203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e689e155-cda7-4498-8e93-d174ff99fcfe None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:45.050462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e689e155-cda7-4498-8e93-d174ff99fcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:45.096463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:45.156000 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:45.309855 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97529b6-9367-4563-8700-421c4b4acd5a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:45.311474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e97529b6-9367-4563-8700-421c4b4acd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:45.466669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97529b6-9367-4563-8700-421c4b4acd5a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:45.522968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:45.585478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c2fe8-66b7-43fb-b6c1-a4c6e27d0bc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:45.586581 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c2fe8-66b7-43fb-b6c1-a4c6e27d0bc3 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:45.586863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c2fe8-66b7-43fb-b6c1-a4c6e27d0bc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:45.587136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c2fe8-66b7-43fb-b6c1-a4c6e27d0bc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:45.589405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-965c2fe8-66b7-43fb-b6c1-a4c6e27d0bc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:45.683830 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c2fe8-66b7-43fb-b6c1-a4c6e27d0bc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:45.685526 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c2fe8-66b7-43fb-b6c1-a4c6e27d0bc3 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:45.706062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 724/1437] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:46.610585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028c70e-4183-44e2-9bfa-f29e2fffb197 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:46.611364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028c70e-4183-44e2-9bfa-f29e2fffb197 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:46.611364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028c70e-4183-44e2-9bfa-f29e2fffb197 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:46.612650 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:46.618196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826972-75a9-43ea-a777-f1a99a2f8917 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:46.618777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826972-75a9-43ea-a777-f1a99a2f8917 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:46.618899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826972-75a9-43ea-a777-f1a99a2f8917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:46.695476 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-29826972-75a9-43ea-a777-f1a99a2f8917 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:46.696717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-29826972-75a9-43ea-a777-f1a99a2f8917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:46.699408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:46.798150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:46.947383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028c70e-4183-44e2-9bfa-f29e2fffb197 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:46.949461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0028c70e-4183-44e2-9bfa-f29e2fffb197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:47.108075 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028c70e-4183-44e2-9bfa-f29e2fffb197 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:47.138290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:47.234610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f06d6-a7e5-4ce0-913d-1d2c6af2495d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:47.235223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f06d6-a7e5-4ce0-913d-1d2c6af2495d service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:47.235497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f06d6-a7e5-4ce0-913d-1d2c6af2495d service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:47.235799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f06d6-a7e5-4ce0-913d-1d2c6af2495d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:47.237407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b79f06d6-a7e5-4ce0-913d-1d2c6af2495d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:47.334804 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f06d6-a7e5-4ce0-913d-1d2c6af2495d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:47.336682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f06d6-a7e5-4ce0-913d-1d2c6af2495d service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:47.358414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 715/1440] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:49.218238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44fdd1-9651-4332-8232-ee7268f626bb None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:49.218760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44fdd1-9651-4332-8232-ee7268f626bb None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:49.218760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44fdd1-9651-4332-8232-ee7268f626bb None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:49.220126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:49.226873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944a73-8fd8-410f-91a4-4955b04bdd70 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:49.227327 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944a73-8fd8-410f-91a4-4955b04bdd70 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:49.227434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944a73-8fd8-410f-91a4-4955b04bdd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:49.270233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:49.296804 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6b944a73-8fd8-410f-91a4-4955b04bdd70 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:49.298036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6b944a73-8fd8-410f-91a4-4955b04bdd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:49.404406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:49.696095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44fdd1-9651-4332-8232-ee7268f626bb None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:49.697642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0c44fdd1-9651-4332-8232-ee7268f626bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:49.816135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44fdd1-9651-4332-8232-ee7268f626bb None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:49.849404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:49.892340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afb398-288f-4021-9650-623437220721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:49.892945 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afb398-288f-4021-9650-623437220721 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:49.893227 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afb398-288f-4021-9650-623437220721 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:49.893498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afb398-288f-4021-9650-623437220721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:49.895129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-54afb398-288f-4021-9650-623437220721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:50.008486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afb398-288f-4021-9650-623437220721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:50.009856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afb398-288f-4021-9650-623437220721 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:50.028489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 727/1443] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:52.159326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c3e4-a5f0-4716-810d-c1c358594b87 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:52.159959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c3e4-a5f0-4716-810d-c1c358594b87 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:52.159959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c3e4-a5f0-4716-810d-c1c358594b87 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:52.161418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:52.166852 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa89d-8bb3-4918-804a-9f15d12560a8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:52.167875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa89d-8bb3-4918-804a-9f15d12560a8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:52.167991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa89d-8bb3-4918-804a-9f15d12560a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:52.241631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c8cfa89d-8bb3-4918-804a-9f15d12560a8 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:52.242869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c8cfa89d-8bb3-4918-804a-9f15d12560a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:52.256082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:52.342290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:52.632296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c3e4-a5f0-4716-810d-c1c358594b87 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:52.633837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f1f0c3e4-a5f0-4716-810d-c1c358594b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:52.745096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c3e4-a5f0-4716-810d-c1c358594b87 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:52.758493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:52.831338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7b649-fbe8-487a-8b0c-2d0db33ffd52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:52.831987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7b649-fbe8-487a-8b0c-2d0db33ffd52 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:52.832268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7b649-fbe8-487a-8b0c-2d0db33ffd52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:52.832542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7b649-fbe8-487a-8b0c-2d0db33ffd52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:52.834160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-74d7b649-fbe8-487a-8b0c-2d0db33ffd52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:52.969734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7b649-fbe8-487a-8b0c-2d0db33ffd52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:52.970075 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7b649-fbe8-487a-8b0c-2d0db33ffd52 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:52.989676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 718/1446] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:55.535844 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897f0ac-4211-457d-ab66-74d03438eb73 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:55.536444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897f0ac-4211-457d-ab66-74d03438eb73 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:55.536444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897f0ac-4211-457d-ab66-74d03438eb73 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:55.537905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:55.544365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edafc9-896d-4758-8e7a-587ba4dacecb None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:55.544917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edafc9-896d-4758-8e7a-587ba4dacecb None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:55.545218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edafc9-896d-4758-8e7a-587ba4dacecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:55.600404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:55.623343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-f7edafc9-896d-4758-8e7a-587ba4dacecb None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:55.625424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f7edafc9-896d-4758-8e7a-587ba4dacecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:55.783988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:56.069887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897f0ac-4211-457d-ab66-74d03438eb73 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:56.071423 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9897f0ac-4211-457d-ab66-74d03438eb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:56.185681 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897f0ac-4211-457d-ab66-74d03438eb73 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:56.226966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:56.268182 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0b01e-4daf-415a-a821-80175d61245d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:56.268786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0b01e-4daf-415a-a821-80175d61245d service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:56.269060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0b01e-4daf-415a-a821-80175d61245d service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:56.269328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0b01e-4daf-415a-a821-80175d61245d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:56.270919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-baa0b01e-4daf-415a-a821-80175d61245d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:56.380352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0b01e-4daf-415a-a821-80175d61245d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:56.381695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0b01e-4daf-415a-a821-80175d61245d service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:56.400087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 730/1449] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:22:58.470662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a8642-93c1-4885-bdbe-83915bdbf446 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:58.471232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a8642-93c1-4885-bdbe-83915bdbf446 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:58.471232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a8642-93c1-4885-bdbe-83915bdbf446 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:58.472492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:22:58.478625 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad76cc-2ad8-40b9-b3cb-156e3c04b6ca None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:58.479176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad76cc-2ad8-40b9-b3cb-156e3c04b6ca None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:58.479176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad76cc-2ad8-40b9-b3cb-156e3c04b6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:58.551127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8bad76cc-2ad8-40b9-b3cb-156e3c04b6ca None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:22:58.560168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8bad76cc-2ad8-40b9-b3cb-156e3c04b6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:58.574300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:58.662651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:22:58.952736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a8642-93c1-4885-bdbe-83915bdbf446 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:22:58.954549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-547a8642-93c1-4885-bdbe-83915bdbf446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:59.075222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a8642-93c1-4885-bdbe-83915bdbf446 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:22:59.096638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:22:59.157074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b82d6-6646-4a8f-a7dd-de994d58bf91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:22:59.157688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b82d6-6646-4a8f-a7dd-de994d58bf91 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:22:59.157964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b82d6-6646-4a8f-a7dd-de994d58bf91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:22:59.158233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b82d6-6646-4a8f-a7dd-de994d58bf91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:22:59.159903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-968b82d6-6646-4a8f-a7dd-de994d58bf91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:22:59.287217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b82d6-6646-4a8f-a7dd-de994d58bf91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:22:59.288730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b82d6-6646-4a8f-a7dd-de994d58bf91 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:22:59.309142 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 721/1452] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:01.437217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca26f4-4948-4773-8e54-2046d2a2f938 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:01.437763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca26f4-4948-4773-8e54-2046d2a2f938 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:01.437763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca26f4-4948-4773-8e54-2046d2a2f938 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:01.439075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:01.444969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a268d3-9f7c-44cd-b913-621ca8fdf538 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:01.445533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a268d3-9f7c-44cd-b913-621ca8fdf538 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:01.445533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a268d3-9f7c-44cd-b913-621ca8fdf538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:01.519136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:01.522302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-01a268d3-9f7c-44cd-b913-621ca8fdf538 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:01.523631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-01a268d3-9f7c-44cd-b913-621ca8fdf538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:01.633212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:01.920682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca26f4-4948-4773-8e54-2046d2a2f938 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:01.922234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ecca26f4-4948-4773-8e54-2046d2a2f938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:02.038272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca26f4-4948-4773-8e54-2046d2a2f938 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:02.093976 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:02.120756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76aa4b5-cf71-4130-b31d-1876b6d8544c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:02.121378 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aa4b5-cf71-4130-b31d-1876b6d8544c service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:02.121655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aa4b5-cf71-4130-b31d-1876b6d8544c service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:02.121927 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aa4b5-cf71-4130-b31d-1876b6d8544c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:02.123624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e76aa4b5-cf71-4130-b31d-1876b6d8544c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:02.358227 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76aa4b5-cf71-4130-b31d-1876b6d8544c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:02.359719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76aa4b5-cf71-4130-b31d-1876b6d8544c service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:02.380820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 733/1455] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:04.384094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c148de-0b5f-41a5-86ae-141f7de17c07 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:04.384674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c148de-0b5f-41a5-86ae-141f7de17c07 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:04.384674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c148de-0b5f-41a5-86ae-141f7de17c07 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:04.386611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:04.391953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c9b6-075d-436d-8012-d47e8f2bb3a3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:04.392377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c9b6-075d-436d-8012-d47e8f2bb3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:04.392580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c9b6-075d-436d-8012-d47e8f2bb3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:04.460580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4803c9b6-075d-436d-8012-d47e8f2bb3a3 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:04.461810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4803c9b6-075d-436d-8012-d47e8f2bb3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:04.492118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:04.564151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:04.850935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c148de-0b5f-41a5-86ae-141f7de17c07 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:04.852528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-78c148de-0b5f-41a5-86ae-141f7de17c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:04.959253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c148de-0b5f-41a5-86ae-141f7de17c07 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:04.989552 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:05.041674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee9639-45d1-48cb-a315-01f41fa9020d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:05.042313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9639-45d1-48cb-a315-01f41fa9020d service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:05.043105 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9639-45d1-48cb-a315-01f41fa9020d service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:05.043381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9639-45d1-48cb-a315-01f41fa9020d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:05.045044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-62ee9639-45d1-48cb-a315-01f41fa9020d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:05.159061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee9639-45d1-48cb-a315-01f41fa9020d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:05.160526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee9639-45d1-48cb-a315-01f41fa9020d service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:05.183716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 724/1458] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:23:07.166882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25627b80-fcf9-48d8-a86a-2a4502ff86bd None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:07.167458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25627b80-fcf9-48d8-a86a-2a4502ff86bd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:07.167458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25627b80-fcf9-48d8-a86a-2a4502ff86bd None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:07.168804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:07.174270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dd8eb-1bb8-4cac-b562-04688192ee71 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:07.174719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dd8eb-1bb8-4cac-b562-04688192ee71 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:07.174825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dd8eb-1bb8-4cac-b562-04688192ee71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:07.246796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ef3dd8eb-1bb8-4cac-b562-04688192ee71 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:07.248071 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ef3dd8eb-1bb8-4cac-b562-04688192ee71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:07.257463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:07.355524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:07.641203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25627b80-fcf9-48d8-a86a-2a4502ff86bd None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:07.642747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-25627b80-fcf9-48d8-a86a-2a4502ff86bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:07.752515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25627b80-fcf9-48d8-a86a-2a4502ff86bd None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:07.811383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:07.830056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d074db-c4ae-473a-9c91-9b2d532f156c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:07.830675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d074db-c4ae-473a-9c91-9b2d532f156c service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:07.830964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d074db-c4ae-473a-9c91-9b2d532f156c service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:07.831229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d074db-c4ae-473a-9c91-9b2d532f156c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:07.832893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-56d074db-c4ae-473a-9c91-9b2d532f156c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:07.956362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d074db-c4ae-473a-9c91-9b2d532f156c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:07.957742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d074db-c4ae-473a-9c91-9b2d532f156c service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:07.977870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 736/1461] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:09.975091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a87939-9fcf-4d09-8757-83fd55973531 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:09.975690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a87939-9fcf-4d09-8757-83fd55973531 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:09.975690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a87939-9fcf-4d09-8757-83fd55973531 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:09.977041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:09.982868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedbee0-7662-4e4f-9477-784890d28e7a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:09.983380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedbee0-7662-4e4f-9477-784890d28e7a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:09.983380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedbee0-7662-4e4f-9477-784890d28e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:10.057799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6fedbee0-7662-4e4f-9477-784890d28e7a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:10.059057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6fedbee0-7662-4e4f-9477-784890d28e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:10.102499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:10.167474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:10.453026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a87939-9fcf-4d09-8757-83fd55973531 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:10.454595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d2a87939-9fcf-4d09-8757-83fd55973531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:10.565698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a87939-9fcf-4d09-8757-83fd55973531 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:10.606039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:10.644094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1bbad-b43e-428b-a58d-ef6e8cc4f4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:10.644708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1bbad-b43e-428b-a58d-ef6e8cc4f4fc service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:10.644984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1bbad-b43e-428b-a58d-ef6e8cc4f4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:10.645252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1bbad-b43e-428b-a58d-ef6e8cc4f4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:10.646890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c5f1bbad-b43e-428b-a58d-ef6e8cc4f4fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:10.762617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1bbad-b43e-428b-a58d-ef6e8cc4f4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:10.764026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1bbad-b43e-428b-a58d-ef6e8cc4f4fc service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:10.790359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 727/1464] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:12.781699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a0f63-c3bc-4e04-830d-a94221f25247 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:12.782274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a0f63-c3bc-4e04-830d-a94221f25247 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:12.782274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a0f63-c3bc-4e04-830d-a94221f25247 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:12.783602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:12.789253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec37927-8a82-483e-8f69-acde7135c9e1 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:12.789948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec37927-8a82-483e-8f69-acde7135c9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:12.789948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec37927-8a82-483e-8f69-acde7135c9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:12.862010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1ec37927-8a82-483e-8f69-acde7135c9e1 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:12.863183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1ec37927-8a82-483e-8f69-acde7135c9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:12.882694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:12.974423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:13.263845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a0f63-c3bc-4e04-830d-a94221f25247 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:13.265483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-692a0f63-c3bc-4e04-830d-a94221f25247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:13.381795 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a0f63-c3bc-4e04-830d-a94221f25247 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:13.465937 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:13.474616 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2218d-db0e-4bf5-ae91-c7c0b600a402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:13.479500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2218d-db0e-4bf5-ae91-c7c0b600a402 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:13.479500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2218d-db0e-4bf5-ae91-c7c0b600a402 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:13.479500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2218d-db0e-4bf5-ae91-c7c0b600a402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:13.479500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-81f2218d-db0e-4bf5-ae91-c7c0b600a402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:13.614972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2218d-db0e-4bf5-ae91-c7c0b600a402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:13.616576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2218d-db0e-4bf5-ae91-c7c0b600a402 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:13.645325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 739/1467] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:15.639846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f1e9f-7636-4037-bf1b-5d3666e54c9c None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:15.640431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f1e9f-7636-4037-bf1b-5d3666e54c9c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:15.640431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f1e9f-7636-4037-bf1b-5d3666e54c9c None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:15.641741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:15.647364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e84e6-4288-4e94-b97f-0d5a397dd9bf None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:15.647870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e84e6-4288-4e94-b97f-0d5a397dd9bf None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:15.647963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e84e6-4288-4e94-b97f-0d5a397dd9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:15.733999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6f1e84e6-4288-4e94-b97f-0d5a397dd9bf None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:15.735216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6f1e84e6-4288-4e94-b97f-0d5a397dd9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:15.787688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:15.843793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:16.129557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f1e9f-7636-4037-bf1b-5d3666e54c9c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:16.131103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4d3f1e9f-7636-4037-bf1b-5d3666e54c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:16.248320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f1e9f-7636-4037-bf1b-5d3666e54c9c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:16.304129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:16.333215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c7ebb-e7d7-4ee4-a285-7103c44b5547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:16.333839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c7ebb-e7d7-4ee4-a285-7103c44b5547 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:16.334112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c7ebb-e7d7-4ee4-a285-7103c44b5547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:16.334379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c7ebb-e7d7-4ee4-a285-7103c44b5547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:16.336208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6d3c7ebb-e7d7-4ee4-a285-7103c44b5547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:16.456822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c7ebb-e7d7-4ee4-a285-7103c44b5547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:16.458282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c7ebb-e7d7-4ee4-a285-7103c44b5547 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:16.479234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 730/1470] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:25.841809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=22 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:26.476009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=66 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:36.443152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111c321-673f-4d2f-a7d8-04a2c87e3f87 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:36.443804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111c321-673f-4d2f-a7d8-04a2c87e3f87 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:36.443804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111c321-673f-4d2f-a7d8-04a2c87e3f87 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:36.445077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 741/1471] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:36.450939 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcf0f3-97dd-4d4e-aa74-4d5ef020ca2f None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:36.451396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcf0f3-97dd-4d4e-aa74-4d5ef020ca2f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:36.451515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcf0f3-97dd-4d4e-aa74-4d5ef020ca2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:36.520202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-49dcf0f3-97dd-4d4e-aa74-4d5ef020ca2f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:36.521418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-49dcf0f3-97dd-4d4e-aa74-4d5ef020ca2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:36.617294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 731/1472] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:36.913141 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13003-f22c-49d0-b7ff-195316ddb6de None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:36.913422 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13003-f22c-49d0-b7ff-195316ddb6de None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:36.913694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13003-f22c-49d0-b7ff-195316ddb6de None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:36.915066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 742/1473] 10.222.0.62 () {54 vars in 961 bytes} [Mon Sep 5 12:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:36.920446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b395e3c-69ce-4a55-b181-206ea6155ab7 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:36.920794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b395e3c-69ce-4a55-b181-206ea6155ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:36.921006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b395e3c-69ce-4a55-b181-206ea6155ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:37.120789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7b395e3c-69ce-4a55-b181-206ea6155ab7 None None] MFA Rules not processed for user `237bbeebe2a949aba444353c1d2ee04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:37.121995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7b395e3c-69ce-4a55-b181-206ea6155ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:37.143515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:37.202188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 732/1474] 10.222.0.62 () {58 vars in 1058 bytes} [Mon Sep 5 12:23:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:37.210742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a13003-f22c-49d0-b7ff-195316ddb6de None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:37.212968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-04a13003-f22c-49d0-b7ff-195316ddb6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:37.321127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a13003-f22c-49d0-b7ff-195316ddb6de None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:37.400257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cccfdf-074a-4c60-a98f-d5c209b6c86a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '237bbeebe2a949aba444353c1d2ee04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:37.400777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cccfdf-074a-4c60-a98f-d5c209b6c86a service glance] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:37.401054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cccfdf-074a-4c60-a98f-d5c209b6c86a service glance] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:37.401326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cccfdf-074a-4c60-a98f-d5c209b6c86a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:37.402941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e5cccfdf-074a-4c60-a98f-d5c209b6c86a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:37.459009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:37.514134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cccfdf-074a-4c60-a98f-d5c209b6c86a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:37.515582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cccfdf-074a-4c60-a98f-d5c209b6c86a service glance] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:37.534135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 743/1475] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:39.551921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a51f45-f29a-45c8-911e-0c355d8e6450 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:39.552462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a51f45-f29a-45c8-911e-0c355d8e6450 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:39.552462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a51f45-f29a-45c8-911e-0c355d8e6450 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:39.555874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 733/1476] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:39.559672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b558b-71f0-4a7c-8077-2f9532020ea7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:39.560174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b558b-71f0-4a7c-8077-2f9532020ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:39.560300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b558b-71f0-4a7c-8077-2f9532020ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:39.631871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-362b558b-71f0-4a7c-8077-2f9532020ea7 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:39.633084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-362b558b-71f0-4a7c-8077-2f9532020ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:39.732635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 744/1477] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:40.008223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a51f45-f29a-45c8-911e-0c355d8e6450 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:40.010196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-43a51f45-f29a-45c8-911e-0c355d8e6450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:40.153267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a51f45-f29a-45c8-911e-0c355d8e6450 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:40.207925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:40.257601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af36e0e-a50c-4e51-9bb1-5c2efdf517cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:40.258217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af36e0e-a50c-4e51-9bb1-5c2efdf517cd admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:40.258569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af36e0e-a50c-4e51-9bb1-5c2efdf517cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:40.258839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af36e0e-a50c-4e51-9bb1-5c2efdf517cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:40.260136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af36e0e-a50c-4e51-9bb1-5c2efdf517cd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:40.261938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WARNING py.warnings [None req-3af36e0e-a50c-4e51-9bb1-5c2efdf517cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:23:40.261938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: warnings.warn(msg) Sep 05 12:23:40.261938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: Sep 05 12:23:40.263139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af36e0e-a50c-4e51-9bb1-5c2efdf517cd admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:40.283119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 734/1478] 10.222.0.62 () {56 vars in 1108 bytes} [Mon Sep 5 12:23:40 2022] GET /identity/v3/projects => generated 2553 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:23:42.234131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87795b-a848-45f9-bbdc-42d00fdcc584 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:42.234716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87795b-a848-45f9-bbdc-42d00fdcc584 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:42.234716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87795b-a848-45f9-bbdc-42d00fdcc584 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:42.236056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 745/1479] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:42.242005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573d64-09eb-435f-9896-cd0ad1730cc4 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:42.242462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573d64-09eb-435f-9896-cd0ad1730cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:42.242613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573d64-09eb-435f-9896-cd0ad1730cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:42.322166 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-eb573d64-09eb-435f-9896-cd0ad1730cc4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:42.323366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-eb573d64-09eb-435f-9896-cd0ad1730cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:42.440382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 735/1480] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:42.737112 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87795b-a848-45f9-bbdc-42d00fdcc584 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:42.738743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1a87795b-a848-45f9-bbdc-42d00fdcc584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:42.749723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:42.905673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87795b-a848-45f9-bbdc-42d00fdcc584 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:43.002179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:43.044832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc442db-80d6-4f9a-a1e4-83acd8c03c63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:43.045906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc442db-80d6-4f9a-a1e4-83acd8c03c63 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:43.046174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc442db-80d6-4f9a-a1e4-83acd8c03c63 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:43.046444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc442db-80d6-4f9a-a1e4-83acd8c03c63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:43.049082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2cc442db-80d6-4f9a-a1e4-83acd8c03c63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:43.155709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc442db-80d6-4f9a-a1e4-83acd8c03c63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:43.157465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc442db-80d6-4f9a-a1e4-83acd8c03c63 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:43.178022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 746/1481] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:45.354008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec97b1a-8502-4d89-8a54-5a8bf3ed3996 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:45.354530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec97b1a-8502-4d89-8a54-5a8bf3ed3996 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:45.354530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec97b1a-8502-4d89-8a54-5a8bf3ed3996 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:45.356693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 736/1482] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:23:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:45.362486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd3650-68d6-4366-8052-867fbf704d28 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:45.363052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd3650-68d6-4366-8052-867fbf704d28 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:45.363052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd3650-68d6-4366-8052-867fbf704d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:45.435672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2cdd3650-68d6-4366-8052-867fbf704d28 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:45.437036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2cdd3650-68d6-4366-8052-867fbf704d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:45.487171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:45.541877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 747/1483] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:45.830995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec97b1a-8502-4d89-8a54-5a8bf3ed3996 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:45.832554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fec97b1a-8502-4d89-8a54-5a8bf3ed3996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:45.857198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:45.945768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec97b1a-8502-4d89-8a54-5a8bf3ed3996 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:46.027394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4c325-d310-4ecb-9e4f-dd4a7613f7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:46.028032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4c325-d310-4ecb-9e4f-dd4a7613f7ad service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:46.028306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4c325-d310-4ecb-9e4f-dd4a7613f7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:46.028570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4c325-d310-4ecb-9e4f-dd4a7613f7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:46.030155 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-23a4c325-d310-4ecb-9e4f-dd4a7613f7ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:46.109330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:46.144189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4c325-d310-4ecb-9e4f-dd4a7613f7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:46.145778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4c325-d310-4ecb-9e4f-dd4a7613f7ad service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:46.164062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 737/1484] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:48.167846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788deba-47a7-4383-882b-557dbd78846f None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:48.168412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788deba-47a7-4383-882b-557dbd78846f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:48.168412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788deba-47a7-4383-882b-557dbd78846f None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:48.169725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 748/1485] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:48.174978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5deb-de89-4cee-a3d2-2ecb4119a7dd None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:48.175438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5deb-de89-4cee-a3d2-2ecb4119a7dd None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:48.175614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5deb-de89-4cee-a3d2-2ecb4119a7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:48.246027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-f3bd5deb-de89-4cee-a3d2-2ecb4119a7dd None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:48.247278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f3bd5deb-de89-4cee-a3d2-2ecb4119a7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:48.354960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 738/1486] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:48.644711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788deba-47a7-4383-882b-557dbd78846f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:48.646289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3788deba-47a7-4383-882b-557dbd78846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:48.755552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788deba-47a7-4383-882b-557dbd78846f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:48.839724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:48.841816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b82202-3aec-4e19-a4f8-743e929ee73a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:48.842528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b82202-3aec-4e19-a4f8-743e929ee73a service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:48.843084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b82202-3aec-4e19-a4f8-743e929ee73a service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:48.843455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b82202-3aec-4e19-a4f8-743e929ee73a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:48.845388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b3b82202-3aec-4e19-a4f8-743e929ee73a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:48.966367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b82202-3aec-4e19-a4f8-743e929ee73a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:48.967875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b82202-3aec-4e19-a4f8-743e929ee73a service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:48.987916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 749/1487] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:51.055616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd948561-e694-40a5-84c1-de010abb0af5 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:51.056158 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd948561-e694-40a5-84c1-de010abb0af5 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:51.056158 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd948561-e694-40a5-84c1-de010abb0af5 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:51.057521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 739/1488] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:51.063630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c568e17-d414-4e8a-9842-085dc88bb347 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:51.064135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c568e17-d414-4e8a-9842-085dc88bb347 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:51.064255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c568e17-d414-4e8a-9842-085dc88bb347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:51.143668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7c568e17-d414-4e8a-9842-085dc88bb347 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:51.144920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7c568e17-d414-4e8a-9842-085dc88bb347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:51.199491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:51.246713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 750/1489] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:51.559600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd948561-e694-40a5-84c1-de010abb0af5 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:51.561154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cd948561-e694-40a5-84c1-de010abb0af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:51.598017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:51.669058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd948561-e694-40a5-84c1-de010abb0af5 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:51.744481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996161e-753a-4af6-97ee-1d3886cef356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:51.745086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996161e-753a-4af6-97ee-1d3886cef356 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:51.745360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996161e-753a-4af6-97ee-1d3886cef356 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:51.745621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996161e-753a-4af6-97ee-1d3886cef356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:51.747234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7996161e-753a-4af6-97ee-1d3886cef356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:51.828993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:51.858392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996161e-753a-4af6-97ee-1d3886cef356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:51.860196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996161e-753a-4af6-97ee-1d3886cef356 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:51.879635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 740/1490] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:23:54.377440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827c92-5990-4e2d-bef9-4c61d1891834 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:54.378044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827c92-5990-4e2d-bef9-4c61d1891834 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:54.378044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827c92-5990-4e2d-bef9-4c61d1891834 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:54.379298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 751/1491] 10.222.0.62 () {54 vars in 927 bytes} [Mon Sep 5 12:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:23:54.384765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201477-7a26-4d2c-9960-f19d558a1095 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:54.385223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201477-7a26-4d2c-9960-f19d558a1095 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:54.385327 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201477-7a26-4d2c-9960-f19d558a1095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:54.454026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-9a201477-7a26-4d2c-9960-f19d558a1095 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:23:54.455284 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9a201477-7a26-4d2c-9960-f19d558a1095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:54.559587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 741/1492] 10.222.0.62 () {58 vars in 1024 bytes} [Mon Sep 5 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:23:54.848339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21827c92-5990-4e2d-bef9-4c61d1891834 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:23:54.849866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-21827c92-5990-4e2d-bef9-4c61d1891834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:54.964181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21827c92-5990-4e2d-bef9-4c61d1891834 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:23:55.043097 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88540a94-4ef5-49b8-9dc3-1e9980cca2a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:23:55.043738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88540a94-4ef5-49b8-9dc3-1e9980cca2a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:23:55.044076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88540a94-4ef5-49b8-9dc3-1e9980cca2a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:23:55.044354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88540a94-4ef5-49b8-9dc3-1e9980cca2a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:23:55.045958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-88540a94-4ef5-49b8-9dc3-1e9980cca2a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:23:55.056567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:23:55.163189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88540a94-4ef5-49b8-9dc3-1e9980cca2a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:23:55.165233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88540a94-4ef5-49b8-9dc3-1e9980cca2a7 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:23:55.184413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 752/1493] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:24:04.560445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=82 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:24:05.182019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=50 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:33.303165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f508-bd6d-48e5-8e26-d51699bc6aac None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:33.303774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f508-bd6d-48e5-8e26-d51699bc6aac None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:33.303774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f508-bd6d-48e5-8e26-d51699bc6aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:33.383915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-5204f508-bd6d-48e5-8e26-d51699bc6aac None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:33.385143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5204f508-bd6d-48e5-8e26-d51699bc6aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:33.486943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 742/1494] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:25:33.495455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334d734-56d2-4d33-a376-77102a3c55b5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:33.496061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334d734-56d2-4d33-a376-77102a3c55b5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:33.496061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334d734-56d2-4d33-a376-77102a3c55b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:33.576736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4334d734-56d2-4d33-a376-77102a3c55b5 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:33.577873 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4334d734-56d2-4d33-a376-77102a3c55b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:33.677819 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 753/1495] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:25:33.683084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204f508-bd6d-48e5-8e26-d51699bc6aac None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:33.684414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5204f508-bd6d-48e5-8e26-d51699bc6aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:33.696068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204f508-bd6d-48e5-8e26-d51699bc6aac None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:33.759024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:33.794841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:33.795446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:33.795749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:33.796017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:33.796655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:25:33.796997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:33.798640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WARNING py.warnings [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:25:33.798640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: warnings.warn(msg) Sep 05 12:25:33.798640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: Sep 05 12:25:33.799688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:33.812076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 743/1496] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:25:33.820343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334d734-56d2-4d33-a376-77102a3c55b5 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:33.829926 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334d734-56d2-4d33-a376-77102a3c55b5 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:33.894015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:33.930260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700f55e-3bb1-47f2-b755-cbdbb52b8088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:33.930870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700f55e-3bb1-47f2-b755-cbdbb52b8088 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:33.931143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700f55e-3bb1-47f2-b755-cbdbb52b8088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:33.931409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700f55e-3bb1-47f2-b755-cbdbb52b8088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:33.932077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700f55e-3bb1-47f2-b755-cbdbb52b8088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:33.933839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-1700f55e-3bb1-47f2-b755-cbdbb52b8088 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:25:33.933839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Sep 05 12:25:33.933839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: Sep 05 12:25:33.935037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700f55e-3bb1-47f2-b755-cbdbb52b8088 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:34.081128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 754/1497] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:25:33 2022] POST /identity/v3/projects => generated 379 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:25:34.086432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:34.088118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:34.124609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e49ff-5516-4b4f-a83f-83d2e58dfccc admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:34.225423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b4a93-e836-4e17-91a3-7c7f246716ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:34.226029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b4a93-e836-4e17-91a3-7c7f246716ba admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:34.226334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b4a93-e836-4e17-91a3-7c7f246716ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:34.226605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b4a93-e836-4e17-91a3-7c7f246716ba admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:34.227245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b4a93-e836-4e17-91a3-7c7f246716ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:34.228338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WARNING py.warnings [None req-e90b4a93-e836-4e17-91a3-7c7f246716ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:25:34.228338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: warnings.warn(msg) Sep 05 12:25:34.228338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: Sep 05 12:25:34.229559 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b4a93-e836-4e17-91a3-7c7f246716ba admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:34.365591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 744/1498] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:25:34 2022] POST /identity/v3/users => generated 290 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:25:34.369700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700f55e-3bb1-47f2-b755-cbdbb52b8088 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:34.371270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1700f55e-3bb1-47f2-b755-cbdbb52b8088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:34.436107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700f55e-3bb1-47f2-b755-cbdbb52b8088 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:34.523315 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:34.534468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf623b3-c2d1-44d9-9808-ff8c91e49202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:34.539033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf623b3-c2d1-44d9-9808-ff8c91e49202 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:34.539033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf623b3-c2d1-44d9-9808-ff8c91e49202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:34.539033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf623b3-c2d1-44d9-9808-ff8c91e49202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:34.539033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf623b3-c2d1-44d9-9808-ff8c91e49202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:34.539033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-caf623b3-c2d1-44d9-9808-ff8c91e49202 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:25:34.539033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Sep 05 12:25:34.539033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: Sep 05 12:25:34.539637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf623b3-c2d1-44d9-9808-ff8c91e49202 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:34.553509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 755/1499] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:25:34.558234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b4a93-e836-4e17-91a3-7c7f246716ba admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:34.562035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e90b4a93-e836-4e17-91a3-7c7f246716ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:34.572305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b4a93-e836-4e17-91a3-7c7f246716ba admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:34.601199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:34.676982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae430b26-165e-47c4-8df7-e82fd618357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:34.677591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430b26-165e-47c4-8df7-e82fd618357f admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:34.677869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430b26-165e-47c4-8df7-e82fd618357f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:34.678143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430b26-165e-47c4-8df7-e82fd618357f admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:34.678793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae430b26-165e-47c4-8df7-e82fd618357f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:34.680361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WARNING py.warnings [None req-ae430b26-165e-47c4-8df7-e82fd618357f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:25:34.680361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: warnings.warn(msg) Sep 05 12:25:34.680361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: Sep 05 12:25:34.681256 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae430b26-165e-47c4-8df7-e82fd618357f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:34.694394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 745/1500] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:25:34.699231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf623b3-c2d1-44d9-9808-ff8c91e49202 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:34.700806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-caf623b3-c2d1-44d9-9808-ff8c91e49202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:34.734509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf623b3-c2d1-44d9-9808-ff8c91e49202 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:34.831348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f7f65-e176-42ac-b1e5-cd22781c11c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:34.832034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f7f65-e176-42ac-b1e5-cd22781c11c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:34.832307 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f7f65-e176-42ac-b1e5-cd22781c11c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:34.832574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f7f65-e176-42ac-b1e5-cd22781c11c9 admin admin] PATH_INFO: `/v3/projects/5dac105a5b5549969a962b4058c6e135/users/d174b9c4d572492296b669c8cac26bcd/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:34.855888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f7f65-e176-42ac-b1e5-cd22781c11c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dac105a5b5549969a962b4058c6e135, user_id=d174b9c4d572492296b669c8cac26bcd, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:34.856921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-fa8f7f65-e176-42ac-b1e5-cd22781c11c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:25:34.856921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Sep 05 12:25:34.856921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: Sep 05 12:25:34.858929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f7f65-e176-42ac-b1e5-cd22781c11c9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:34.954127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 756/1501] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:25:34 2022] PUT /identity/v3/projects/5dac105a5b5549969a962b4058c6e135/users/d174b9c4d572492296b669c8cac26bcd/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:25:34.966487 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247a332-1d91-4f5d-a3e4-19b78e8d1da8 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:34.966772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247a332-1d91-4f5d-a3e4-19b78e8d1da8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:34.967044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247a332-1d91-4f5d-a3e4-19b78e8d1da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:35.024760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-5247a332-1d91-4f5d-a3e4-19b78e8d1da8 None None] MFA Rules not processed for user `d174b9c4d572492296b669c8cac26bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:35.025959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5247a332-1d91-4f5d-a3e4-19b78e8d1da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:35.105605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 746/1502] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:25:35.119627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f89c6-ff77-4eb8-b8d1-4e6a34ecfc04 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:35.119907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f89c6-ff77-4eb8-b8d1-4e6a34ecfc04 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:35.120175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f89c6-ff77-4eb8-b8d1-4e6a34ecfc04 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:35.121521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 757/1503] 10.222.0.62 () {54 vars in 964 bytes} [Mon Sep 5 12:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:25:35.126541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f5b97-7edd-424f-a733-b6bdccaf0d5a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:35.126818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f5b97-7edd-424f-a733-b6bdccaf0d5a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:35.127090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f5b97-7edd-424f-a733-b6bdccaf0d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:35.158877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:35.206036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-5b6f5b97-7edd-424f-a733-b6bdccaf0d5a None None] MFA Rules not processed for user `8f33f1f3d1ca4436887c2346bd9fa149`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:35.207171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5b6f5b97-7edd-424f-a733-b6bdccaf0d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:35.284495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 747/1504] 10.222.0.62 () {58 vars in 1061 bytes} [Mon Sep 5 12:25:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:25:35.294773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f89c6-ff77-4eb8-b8d1-4e6a34ecfc04 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:35.296355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7d2f89c6-ff77-4eb8-b8d1-4e6a34ecfc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:35.350639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:35.409508 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f89c6-ff77-4eb8-b8d1-4e6a34ecfc04 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:35.485388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea071-00fd-49bb-aff8-45eb69ad26a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:35.485901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea071-00fd-49bb-aff8-45eb69ad26a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:35.486173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea071-00fd-49bb-aff8-45eb69ad26a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:35.486439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea071-00fd-49bb-aff8-45eb69ad26a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:35.488076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-55bea071-00fd-49bb-aff8-45eb69ad26a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:35.588819 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bea071-00fd-49bb-aff8-45eb69ad26a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:35.590306 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bea071-00fd-49bb-aff8-45eb69ad26a3 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:35.594194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:35.610390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 758/1505] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:25:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:25:35.658349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba624a-532b-424e-af89-df8fe0cb920d None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:35.658842 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba624a-532b-424e-af89-df8fe0cb920d None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:35.658969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba624a-532b-424e-af89-df8fe0cb920d None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:35.660177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 748/1506] 10.222.0.62 () {52 vars in 830 bytes} [Mon Sep 5 12:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:25:35.664057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea071-00fd-49bb-aff8-45eb69ad26a3 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:35.665270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-55bea071-00fd-49bb-aff8-45eb69ad26a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:35.770468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea071-00fd-49bb-aff8-45eb69ad26a3 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:35.869624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:35.870274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:35.870553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:35.870829 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] PATH_INFO: `/v3/users/d174b9c4d572492296b669c8cac26bcd` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:35.885275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d174b9c4d572492296b669c8cac26bcd)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:35.886321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:25:35.886321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Sep 05 12:25:35.886321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: Sep 05 12:25:35.887574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:35.899066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:36.111915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd174b9c4d572492296b669c8cac26bcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:36.112359 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd174b9c4d572492296b669c8cac26bcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:36.186512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd174b9c4d572492296b669c8cac26bcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:36.214597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd174b9c4d572492296b669c8cac26bcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:36.215489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd174b9c4d572492296b669c8cac26bcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:36.235726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 759/1507] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:25:35 2022] DELETE /identity/v3/users/d174b9c4d572492296b669c8cac26bcd => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:25:36.240364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a1daa-a412-41b1-bf51-127a996c3498 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:36.240542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a1daa-a412-41b1-bf51-127a996c3498 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:36.240653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a1daa-a412-41b1-bf51-127a996c3498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:36.300873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-854a1daa-a412-41b1-bf51-127a996c3498 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:25:36.302163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-854a1daa-a412-41b1-bf51-127a996c3498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:36.381231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:36.394534 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 749/1508] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:25:36.405685 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:36.406913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:36.511286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27e2ae-fda4-46f7-9685-e5176ef6796b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:36.567732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:36.587278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82738c-2ec8-4e32-af52-47cb83f005c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:36.587933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82738c-2ec8-4e32-af52-47cb83f005c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:36.588212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82738c-2ec8-4e32-af52-47cb83f005c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:36.588486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82738c-2ec8-4e32-af52-47cb83f005c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:36.590108 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cc82738c-2ec8-4e32-af52-47cb83f005c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:36.704633 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82738c-2ec8-4e32-af52-47cb83f005c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:36.706086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82738c-2ec8-4e32-af52-47cb83f005c7 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:36.725621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 760/1509] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:25:37.266049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a1daa-a412-41b1-bf51-127a996c3498 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:25:37.267325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-854a1daa-a412-41b1-bf51-127a996c3498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:25:37.284444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a1daa-a412-41b1-bf51-127a996c3498 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:25:37.398693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:25:37.399331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:25:37.399638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:25:37.399912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] PATH_INFO: `/v3/projects/5dac105a5b5549969a962b4058c6e135` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:25:37.410216 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dac105a5b5549969a962b4058c6e135)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:25:37.411271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WARNING py.warnings [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:25:37.411271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: warnings.warn(msg) Sep 05 12:25:37.411271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: Sep 05 12:25:37.412510 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:25:37.588041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dac105a5b5549969a962b4058c6e135', 'request_id': 'req-2a230c10-f931-4cd5-9afe-c875561600f5'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:37.594364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dac105a5b5549969a962b4058c6e135', 'request_id': 'req-2a230c10-f931-4cd5-9afe-c875561600f5'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:37.683615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dac105a5b5549969a962b4058c6e135', 'request_id': 'req-2a230c10-f931-4cd5-9afe-c875561600f5'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:37.723827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] The token cache is being invalidate because project 5dac105a5b5549969a962b4058c6e135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:25:37.724121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:37.725879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2a230c10-f931-4cd5-9afe-c875561600f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:25:37.727440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 750/1510] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:25:37 2022] DELETE /identity/v3/projects/5dac105a5b5549969a962b4058c6e135 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:25:46.723246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=62 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:25:47.720157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:12.037518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ceb32e-ab7d-4c3f-a559-4297cf2c11da None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:12.038127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ceb32e-ab7d-4c3f-a559-4297cf2c11da None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:12.038127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ceb32e-ab7d-4c3f-a559-4297cf2c11da None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:12.039405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:27:12.044914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd3366-7726-45cf-8086-57b3e3d81e1b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:12.045438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd3366-7726-45cf-8086-57b3e3d81e1b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:12.045438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd3366-7726-45cf-8086-57b3e3d81e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:12.135481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-cddd3366-7726-45cf-8086-57b3e3d81e1b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:12.136993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cddd3366-7726-45cf-8086-57b3e3d81e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:12.246201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:27:12.405444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ceb32e-ab7d-4c3f-a559-4297cf2c11da None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:12.408273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c1ceb32e-ab7d-4c3f-a559-4297cf2c11da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:12.534596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ceb32e-ab7d-4c3f-a559-4297cf2c11da None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:12.540189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba2a2b-56a1-43a0-adbf-5a83da9444ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:12.540785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba2a2b-56a1-43a0-adbf-5a83da9444ea service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:12.541062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba2a2b-56a1-43a0-adbf-5a83da9444ea service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:12.541331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba2a2b-56a1-43a0-adbf-5a83da9444ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:12.549625 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba2a2b-56a1-43a0-adbf-5a83da9444ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:12.551192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba2a2b-56a1-43a0-adbf-5a83da9444ea service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:12.561230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 762/1513] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:13.394872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d134c-0fcd-49d8-83a0-0f7e6cf884e3 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:13.395405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d134c-0fcd-49d8-83a0-0f7e6cf884e3 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:13.395405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d134c-0fcd-49d8-83a0-0f7e6cf884e3 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:13.396681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:27:13.401510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad676c-4c0e-4628-b326-a23a2bf44029 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:13.401968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad676c-4c0e-4628-b326-a23a2bf44029 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:13.402091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad676c-4c0e-4628-b326-a23a2bf44029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:13.423919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e4ad676c-4c0e-4628-b326-a23a2bf44029 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:13.425170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e4ad676c-4c0e-4628-b326-a23a2bf44029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:13.433130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:13.583682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d134c-0fcd-49d8-83a0-0f7e6cf884e3 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:13.598638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d134c-0fcd-49d8-83a0-0f7e6cf884e3 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:13.604073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f3fd7-552b-4c66-b686-e325a7b0ee39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:13.604674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f3fd7-552b-4c66-b686-e325a7b0ee39 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:13.605069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f3fd7-552b-4c66-b686-e325a7b0ee39 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:13.605341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f3fd7-552b-4c66-b686-e325a7b0ee39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:13.614533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f3fd7-552b-4c66-b686-e325a7b0ee39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:13.616154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f3fd7-552b-4c66-b686-e325a7b0ee39 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:13.625709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 753/1516] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:14.166603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00490332-eaa6-462c-a135-e9b17ee3c18b None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:14.166901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00490332-eaa6-462c-a135-e9b17ee3c18b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:14.167170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00490332-eaa6-462c-a135-e9b17ee3c18b None None] PATH_INFO: `/v3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:14.168494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 764/1517] 10.222.0.58 () {54 vars in 925 bytes} [Mon Sep 5 12:27:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:27:14.175213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25dab-f3e0-44e0-ab21-4c1d03c6d8e5 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:14.175497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25dab-f3e0-44e0-ab21-4c1d03c6d8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:14.175795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25dab-f3e0-44e0-ab21-4c1d03c6d8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:14.235737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7ec25dab-f3e0-44e0-ab21-4c1d03c6d8e5 None None] MFA Rules not processed for user `90d65372febb414d82843f9dfa56f738`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:14.236945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7ec25dab-f3e0-44e0-ab21-4c1d03c6d8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:14.254719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:14.320749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 754/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Mon Sep 5 12:27:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:23.418204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=56 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:23.781220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b3441-94cf-477b-9778-e7b6b8424bd7 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:23.781502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b3441-94cf-477b-9778-e7b6b8424bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:23.781797 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b3441-94cf-477b-9778-e7b6b8424bd7 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:23.783155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 765/1519] 10.222.0.62 () {54 vars in 961 bytes} [Mon Sep 5 12:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:27:23.788074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d95ae-63e8-4ace-8755-95672a63bf9d None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:23.788510 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d95ae-63e8-4ace-8755-95672a63bf9d None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:23.788659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d95ae-63e8-4ace-8755-95672a63bf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:23.808822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8d5d95ae-63e8-4ace-8755-95672a63bf9d None None] MFA Rules not processed for user `90d65372febb414d82843f9dfa56f738`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:23.810023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8d5d95ae-63e8-4ace-8755-95672a63bf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:23.818685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 755/1520] 10.222.0.62 () {58 vars in 1058 bytes} [Mon Sep 5 12:27:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:23.827484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b3441-94cf-477b-9778-e7b6b8424bd7 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:23.841042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b3441-94cf-477b-9778-e7b6b8424bd7 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:23.846765 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec0d8d-9f18-4d94-b283-cae10187ff59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:23.847351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0d8d-9f18-4d94-b283-cae10187ff59 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:23.847772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0d8d-9f18-4d94-b283-cae10187ff59 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:23.848051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0d8d-9f18-4d94-b283-cae10187ff59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:23.856113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec0d8d-9f18-4d94-b283-cae10187ff59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:23.857708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec0d8d-9f18-4d94-b283-cae10187ff59 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:23.867999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 766/1521] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:24.863859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db55fba-4d74-4910-8f65-a670f7af2b12 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:24.864427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db55fba-4d74-4910-8f65-a670f7af2b12 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:24.864427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db55fba-4d74-4910-8f65-a670f7af2b12 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:24.865850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:27:24.870957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f926857-ab82-44e7-b4a9-65a288b1d226 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:24.871386 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f926857-ab82-44e7-b4a9-65a288b1d226 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:24.871506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f926857-ab82-44e7-b4a9-65a288b1d226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:24.892698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8f926857-ab82-44e7-b4a9-65a288b1d226 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:24.893968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8f926857-ab82-44e7-b4a9-65a288b1d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:24.902072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:25.056816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db55fba-4d74-4910-8f65-a670f7af2b12 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:25.073720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db55fba-4d74-4910-8f65-a670f7af2b12 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:25.079273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467df768-6c43-4cc6-b496-edad7a1f4033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:25.079930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467df768-6c43-4cc6-b496-edad7a1f4033 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:25.080284 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467df768-6c43-4cc6-b496-edad7a1f4033 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:25.080530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467df768-6c43-4cc6-b496-edad7a1f4033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:25.088659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467df768-6c43-4cc6-b496-edad7a1f4033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:25.090343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467df768-6c43-4cc6-b496-edad7a1f4033 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:25.099937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 757/1524] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:25.919563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ebcf8-5e13-426e-a729-f69c5b493e10 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:25.920196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ebcf8-5e13-426e-a729-f69c5b493e10 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:25.920196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ebcf8-5e13-426e-a729-f69c5b493e10 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:25.921504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 12:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:27:25.926251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67579f9b-f23e-4fc4-823b-5a3192a51347 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:25.926663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67579f9b-f23e-4fc4-823b-5a3192a51347 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:25.926782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67579f9b-f23e-4fc4-823b-5a3192a51347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:25.947842 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-67579f9b-f23e-4fc4-823b-5a3192a51347 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:25.949094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-67579f9b-f23e-4fc4-823b-5a3192a51347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:25.957515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 12:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:26.110682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ebcf8-5e13-426e-a729-f69c5b493e10 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:26.125399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ebcf8-5e13-426e-a729-f69c5b493e10 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:26.130854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb5d70-2089-49c4-a63d-fd669d323031 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:26.131462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb5d70-2089-49c4-a63d-fd669d323031 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:26.131758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb5d70-2089-49c4-a63d-fd669d323031 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:26.132023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb5d70-2089-49c4-a63d-fd669d323031 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:26.140986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb5d70-2089-49c4-a63d-fd669d323031 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:26.142495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb5d70-2089-49c4-a63d-fd669d323031 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:26.151824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 769/1527] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:27:29.977012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043a536-75c7-4564-bf72-11a03140f9e4 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:29.977639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043a536-75c7-4564-bf72-11a03140f9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:29.977639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043a536-75c7-4564-bf72-11a03140f9e4 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:29.978877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:27:29.984240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afb720-f5be-456e-a294-5b8b0b1f82ef None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:29.984713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afb720-f5be-456e-a294-5b8b0b1f82ef None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:29.984821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afb720-f5be-456e-a294-5b8b0b1f82ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:30.004580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e5afb720-f5be-456e-a294-5b8b0b1f82ef None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:30.005763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e5afb720-f5be-456e-a294-5b8b0b1f82ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:30.013531 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:27:30.305470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043a536-75c7-4564-bf72-11a03140f9e4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:30.307202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9043a536-75c7-4564-bf72-11a03140f9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:30.387007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043a536-75c7-4564-bf72-11a03140f9e4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:30.390747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36749693-cbde-4dab-a64f-89800b987a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:30.391355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36749693-cbde-4dab-a64f-89800b987a7e service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:30.391674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36749693-cbde-4dab-a64f-89800b987a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:30.391953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36749693-cbde-4dab-a64f-89800b987a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:30.400863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36749693-cbde-4dab-a64f-89800b987a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:30.402301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36749693-cbde-4dab-a64f-89800b987a7e service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:30.411606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 760/1530] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:32.315248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be2185-5920-490f-8faf-3e9614161727 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:32.315922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be2185-5920-490f-8faf-3e9614161727 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:32.315922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be2185-5920-490f-8faf-3e9614161727 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:32.317069 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:27:32.322394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758e8ea-d8e9-43f9-b1d8-0d2652320a37 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:32.322855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758e8ea-d8e9-43f9-b1d8-0d2652320a37 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:32.322966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758e8ea-d8e9-43f9-b1d8-0d2652320a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:32.342945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1758e8ea-d8e9-43f9-b1d8-0d2652320a37 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:32.344175 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1758e8ea-d8e9-43f9-b1d8-0d2652320a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:32.353295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:27:32.640539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be2185-5920-490f-8faf-3e9614161727 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:32.651049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be2185-5920-490f-8faf-3e9614161727 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:32.654577 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e684c99-fde6-488e-a2ec-2a4c41e6afea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:32.655267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e684c99-fde6-488e-a2ec-2a4c41e6afea service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:32.655559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e684c99-fde6-488e-a2ec-2a4c41e6afea service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:32.655840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e684c99-fde6-488e-a2ec-2a4c41e6afea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:32.665078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e684c99-fde6-488e-a2ec-2a4c41e6afea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:32.666519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e684c99-fde6-488e-a2ec-2a4c41e6afea service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:32.677584 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 772/1533] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:36.089787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3acb6-fdf4-4b61-8713-c6db9848fdc0 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:36.090331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3acb6-fdf4-4b61-8713-c6db9848fdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:36.090331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3acb6-fdf4-4b61-8713-c6db9848fdc0 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:36.091774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:27:36.097647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6ff73-b967-4e33-a2b6-576a995d09a0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:36.098066 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6ff73-b967-4e33-a2b6-576a995d09a0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:36.098197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6ff73-b967-4e33-a2b6-576a995d09a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:36.118208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b4b6ff73-b967-4e33-a2b6-576a995d09a0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:36.119453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b4b6ff73-b967-4e33-a2b6-576a995d09a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:36.128297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:27:36.678859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3acb6-fdf4-4b61-8713-c6db9848fdc0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:36.680583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-84e3acb6-fdf4-4b61-8713-c6db9848fdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:36.744831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:36.758020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3acb6-fdf4-4b61-8713-c6db9848fdc0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:36.764546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde2ae3-253f-43f3-814b-1005bb1808cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:36.764546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde2ae3-253f-43f3-814b-1005bb1808cb service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:36.764546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde2ae3-253f-43f3-814b-1005bb1808cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:36.764546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde2ae3-253f-43f3-814b-1005bb1808cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:36.771661 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde2ae3-253f-43f3-814b-1005bb1808cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:36.773103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde2ae3-253f-43f3-814b-1005bb1808cb service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:36.782952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 763/1536] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:38.720640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b0634-5ac3-42bc-a2d9-143f6a5fa42e None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:38.721610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b0634-5ac3-42bc-a2d9-143f6a5fa42e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:38.721610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b0634-5ac3-42bc-a2d9-143f6a5fa42e None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:38.723469 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 12:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:27:38.728608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3ee26-5782-4fee-a2fd-b22cfbdaa727 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:38.733612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3ee26-5782-4fee-a2fd-b22cfbdaa727 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:38.733612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3ee26-5782-4fee-a2fd-b22cfbdaa727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:38.749829 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-3af3ee26-5782-4fee-a2fd-b22cfbdaa727 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:27:38.751036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3af3ee26-5782-4fee-a2fd-b22cfbdaa727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:38.759177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 12:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:27:39.320835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b0634-5ac3-42bc-a2d9-143f6a5fa42e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:27:39.322520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3d5b0634-5ac3-42bc-a2d9-143f6a5fa42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:27:39.335251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b0634-5ac3-42bc-a2d9-143f6a5fa42e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:27:39.338763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a4883-6ef8-4631-bed0-1367de5bd739 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:27:39.339353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a4883-6ef8-4631-bed0-1367de5bd739 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:27:39.339663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a4883-6ef8-4631-bed0-1367de5bd739 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:27:39.339933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a4883-6ef8-4631-bed0-1367de5bd739 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:27:39.348868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a4883-6ef8-4631-bed0-1367de5bd739 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:27:39.350269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a4883-6ef8-4631-bed0-1367de5bd739 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:27:39.360037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 775/1539] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:27:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:27:48.743077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=11 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:27:49.355550 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=33 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:04.965116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b182aa-2231-475c-9a8c-99af26212afa None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:04.966282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b182aa-2231-475c-9a8c-99af26212afa None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:04.966799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b182aa-2231-475c-9a8c-99af26212afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.005002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e9b182aa-2231-475c-9a8c-99af26212afa None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.006595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e9b182aa-2231-475c-9a8c-99af26212afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.016363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 765/1540] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.024737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db196-3c65-477c-8c60-9ed1d17bb04d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.025345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db196-3c65-477c-8c60-9ed1d17bb04d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.025345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db196-3c65-477c-8c60-9ed1d17bb04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.057095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ab8db196-3c65-477c-8c60-9ed1d17bb04d None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.058163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ab8db196-3c65-477c-8c60-9ed1d17bb04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.077559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 776/1541] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.082222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b182aa-2231-475c-9a8c-99af26212afa None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.106631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b182aa-2231-475c-9a8c-99af26212afa None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.114018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a762f-1051-4ba8-9e00-d4657cde76db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.115049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a762f-1051-4ba8-9e00-d4657cde76db admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.115635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a762f-1051-4ba8-9e00-d4657cde76db admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.116156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a762f-1051-4ba8-9e00-d4657cde76db admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.117034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a762f-1051-4ba8-9e00-d4657cde76db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:05.117569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a762f-1051-4ba8-9e00-d4657cde76db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.119457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a762f-1051-4ba8-9e00-d4657cde76db admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.137016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 766/1542] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:05.142832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286911f-4726-4125-9a31-f57d070b77f8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.143475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286911f-4726-4125-9a31-f57d070b77f8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.144204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286911f-4726-4125-9a31-f57d070b77f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.145955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a762f-1051-4ba8-9e00-d4657cde76db admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.166906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a762f-1051-4ba8-9e00-d4657cde76db admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.181920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405be71f-937c-4deb-a51e-d8b22b9a650a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.183505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405be71f-937c-4deb-a51e-d8b22b9a650a admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.184156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405be71f-937c-4deb-a51e-d8b22b9a650a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.184831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405be71f-937c-4deb-a51e-d8b22b9a650a admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.185774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405be71f-937c-4deb-a51e-d8b22b9a650a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.187111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WARNING py.warnings [None req-405be71f-937c-4deb-a51e-d8b22b9a650a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:29:05.187111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: warnings.warn(msg) Sep 05 12:29:05.187111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: Sep 05 12:29:05.191276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405be71f-937c-4deb-a51e-d8b22b9a650a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.196005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e286911f-4726-4125-9a31-f57d070b77f8 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.198040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e286911f-4726-4125-9a31-f57d070b77f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.218947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 777/1543] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.246471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4556ba-cd88-4537-81f6-5ece4ce17715 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.247260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4556ba-cd88-4537-81f6-5ece4ce17715 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.247796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4556ba-cd88-4537-81f6-5ece4ce17715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.273110 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4e4556ba-cd88-4537-81f6-5ece4ce17715 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.278291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4e4556ba-cd88-4537-81f6-5ece4ce17715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.284307 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 778/1544] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:29:05.289731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4556ba-cd88-4537-81f6-5ece4ce17715 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.301300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4556ba-cd88-4537-81f6-5ece4ce17715 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.307006 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b705e1-fc31-451a-842b-b674ed6ebfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.311699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b705e1-fc31-451a-842b-b674ed6ebfa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.312089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b705e1-fc31-451a-842b-b674ed6ebfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.312366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b705e1-fc31-451a-842b-b674ed6ebfa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.313091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b705e1-fc31-451a-842b-b674ed6ebfa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:05.313395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b705e1-fc31-451a-842b-b674ed6ebfa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.314450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-90b705e1-fc31-451a-842b-b674ed6ebfa4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:29:05.314450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Sep 05 12:29:05.314450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: Sep 05 12:29:05.315412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b705e1-fc31-451a-842b-b674ed6ebfa4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.330142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 779/1545] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:05.338668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b705e1-fc31-451a-842b-b674ed6ebfa4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.350073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b705e1-fc31-451a-842b-b674ed6ebfa4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.357804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38deeba8-49bf-40e0-af5b-9884b7003ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.363110 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deeba8-49bf-40e0-af5b-9884b7003ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.363430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deeba8-49bf-40e0-af5b-9884b7003ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.363689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deeba8-49bf-40e0-af5b-9884b7003ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.364900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38deeba8-49bf-40e0-af5b-9884b7003ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.368363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 767/1546] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/projects => generated 405 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:05.369775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19d9f3-13ef-4856-9730-b548d37acfe7 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.370116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19d9f3-13ef-4856-9730-b548d37acfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.370397 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19d9f3-13ef-4856-9730-b548d37acfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.374303 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38deeba8-49bf-40e0-af5b-9884b7003ce0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.406509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ec19d9f3-13ef-4856-9730-b548d37acfe7 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.407992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ec19d9f3-13ef-4856-9730-b548d37acfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.503166 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 768/1547] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.504054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19d9f3-13ef-4856-9730-b548d37acfe7 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.506509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 780/1548] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:05.507893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17c81e-d0b6-4c8d-8ff0-efca5eb43d37 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.508283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17c81e-d0b6-4c8d-8ff0-efca5eb43d37 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.508564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17c81e-d0b6-4c8d-8ff0-efca5eb43d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.523347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19d9f3-13ef-4856-9730-b548d37acfe7 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.531727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-da17c81e-d0b6-4c8d-8ff0-efca5eb43d37 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.533068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-da17c81e-d0b6-4c8d-8ff0-efca5eb43d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.587297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad80dd3-76b5-48fa-9e91-3fe265c35de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.588072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad80dd3-76b5-48fa-9e91-3fe265c35de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.588339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad80dd3-76b5-48fa-9e91-3fe265c35de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.588608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad80dd3-76b5-48fa-9e91-3fe265c35de8 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.589266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad80dd3-76b5-48fa-9e91-3fe265c35de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.591242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad80dd3-76b5-48fa-9e91-3fe265c35de8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.604408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 781/1549] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.605662 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc2d31-9566-451a-a032-3e505733729a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.605948 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc2d31-9566-451a-a032-3e505733729a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.606379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc2d31-9566-451a-a032-3e505733729a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.630611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-21dc2d31-9566-451a-a032-3e505733729a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.631888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-21dc2d31-9566-451a-a032-3e505733729a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.639938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 782/1550] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.641152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a4214-56a4-4f9b-8898-47cbbb350396 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.641435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a4214-56a4-4f9b-8898-47cbbb350396 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.641709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a4214-56a4-4f9b-8898-47cbbb350396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.662435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-fa6a4214-56a4-4f9b-8898-47cbbb350396 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.663669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fa6a4214-56a4-4f9b-8898-47cbbb350396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.671511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 769/1551] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/users => generated 303 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:05.671796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 783/1552] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.673094 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5e695-20b7-4469-8792-f3e08ce39714 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.673381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5e695-20b7-4469-8792-f3e08ce39714 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.673654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5e695-20b7-4469-8792-f3e08ce39714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.675446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad80dd3-76b5-48fa-9e91-3fe265c35de8 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.687028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad80dd3-76b5-48fa-9e91-3fe265c35de8 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.691904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325f3f2-242a-407e-897f-6b6b27bd7e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.693012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325f3f2-242a-407e-897f-6b6b27bd7e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.693290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325f3f2-242a-407e-897f-6b6b27bd7e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.693564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325f3f2-242a-407e-897f-6b6b27bd7e5b admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.694196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325f3f2-242a-407e-897f-6b6b27bd7e5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.695444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-eba5e695-20b7-4469-8792-f3e08ce39714 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.696210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325f3f2-242a-407e-897f-6b6b27bd7e5b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.696710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-eba5e695-20b7-4469-8792-f3e08ce39714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.710478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 784/1553] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.710478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157668c-89e2-4157-90fc-f0299418b41b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.710478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157668c-89e2-4157-90fc-f0299418b41b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.710478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157668c-89e2-4157-90fc-f0299418b41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.730109 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f157668c-89e2-4157-90fc-f0299418b41b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.731314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f157668c-89e2-4157-90fc-f0299418b41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.739384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 785/1554] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.740760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc30300-f807-4601-841c-de8c043dcf5a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.741040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc30300-f807-4601-841c-de8c043dcf5a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.741313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc30300-f807-4601-841c-de8c043dcf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.761838 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5cc30300-f807-4601-841c-de8c043dcf5a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.763102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5cc30300-f807-4601-841c-de8c043dcf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.772017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 786/1555] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.773432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd857d-4968-4e2f-97fc-4e5ca9d6d499 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.773761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd857d-4968-4e2f-97fc-4e5ca9d6d499 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.774032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd857d-4968-4e2f-97fc-4e5ca9d6d499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.790251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 770/1556] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:05.790251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707dbdc-32d3-41e4-8527-5a99c073696e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.799440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707dbdc-32d3-41e4-8527-5a99c073696e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.799440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707dbdc-32d3-41e4-8527-5a99c073696e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.799653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9ccd857d-4968-4e2f-97fc-4e5ca9d6d499 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.799653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9ccd857d-4968-4e2f-97fc-4e5ca9d6d499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.804895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 787/1557] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.809621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd857d-4968-4e2f-97fc-4e5ca9d6d499 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.816002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd857d-4968-4e2f-97fc-4e5ca9d6d499 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.816183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d707dbdc-32d3-41e4-8527-5a99c073696e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.817780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d707dbdc-32d3-41e4-8527-5a99c073696e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.821290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996002f-08cc-49fc-828e-55187e50ace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.821903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996002f-08cc-49fc-828e-55187e50ace0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.822184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996002f-08cc-49fc-828e-55187e50ace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.822455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996002f-08cc-49fc-828e-55187e50ace0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.823100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996002f-08cc-49fc-828e-55187e50ace0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.824736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996002f-08cc-49fc-828e-55187e50ace0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.826495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 771/1558] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.827906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ad7c2-0c7d-49d3-9d8c-4380aee1ae7a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.828188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ad7c2-0c7d-49d3-9d8c-4380aee1ae7a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.828458 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ad7c2-0c7d-49d3-9d8c-4380aee1ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.838553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 788/1559] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:05.838553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996002f-08cc-49fc-828e-55187e50ace0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.849770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996002f-08cc-49fc-828e-55187e50ace0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.852784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c57ad7c2-0c7d-49d3-9d8c-4380aee1ae7a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.854090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c57ad7c2-0c7d-49d3-9d8c-4380aee1ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.854546 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eceee3f-65fa-43bc-bc79-62822a0d1a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.855233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eceee3f-65fa-43bc-bc79-62822a0d1a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.855512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eceee3f-65fa-43bc-bc79-62822a0d1a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.856059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eceee3f-65fa-43bc-bc79-62822a0d1a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.856536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eceee3f-65fa-43bc-bc79-62822a0d1a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:05.856835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eceee3f-65fa-43bc-bc79-62822a0d1a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.858436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eceee3f-65fa-43bc-bc79-62822a0d1a88 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.862852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 772/1560] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.864141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82bb67-0b3b-4e4e-8b9e-25bde985a6c4 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.864422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82bb67-0b3b-4e4e-8b9e-25bde985a6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.864715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82bb67-0b3b-4e4e-8b9e-25bde985a6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.871813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 789/1561] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:05.880751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cd29-705f-405a-8e1b-6ebfe7d955ea None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.881034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cd29-705f-405a-8e1b-6ebfe7d955ea None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.881308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cd29-705f-405a-8e1b-6ebfe7d955ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.890743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-db82bb67-0b3b-4e4e-8b9e-25bde985a6c4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.892004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-db82bb67-0b3b-4e4e-8b9e-25bde985a6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.900851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 773/1562] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.901717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82bb67-0b3b-4e4e-8b9e-25bde985a6c4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.901846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7981cd29-705f-405a-8e1b-6ebfe7d955ea None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:05.902963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7981cd29-705f-405a-8e1b-6ebfe7d955ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:05.906531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:05.911214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 790/1563] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:05.912136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981cd29-705f-405a-8e1b-6ebfe7d955ea None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.912407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82bb67-0b3b-4e4e-8b9e-25bde985a6c4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.917620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9e6bd-58ab-4e52-854b-e8ace4a03e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.918248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9e6bd-58ab-4e52-854b-e8ace4a03e61 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.918622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9e6bd-58ab-4e52-854b-e8ace4a03e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.918896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9e6bd-58ab-4e52-854b-e8ace4a03e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.919565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9e6bd-58ab-4e52-854b-e8ace4a03e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.921197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9e6bd-58ab-4e52-854b-e8ace4a03e61 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.922011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981cd29-705f-405a-8e1b-6ebfe7d955ea None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.927169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac9716-da5b-4446-ac74-7fdce7b8c833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.927818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac9716-da5b-4446-ac74-7fdce7b8c833 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.928251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac9716-da5b-4446-ac74-7fdce7b8c833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.928549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac9716-da5b-4446-ac74-7fdce7b8c833 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.929198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac9716-da5b-4446-ac74-7fdce7b8c833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:05.929504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac9716-da5b-4446-ac74-7fdce7b8c833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.931086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac9716-da5b-4446-ac74-7fdce7b8c833 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.932886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 774/1564] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:05.933503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9e6bd-58ab-4e52-854b-e8ace4a03e61 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.945693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9e6bd-58ab-4e52-854b-e8ace4a03e61 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.955750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 791/1565] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:05.955750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac9716-da5b-4446-ac74-7fdce7b8c833 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.956017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a8686-423a-42ea-aaef-57c31b10a457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.956017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a8686-423a-42ea-aaef-57c31b10a457 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.956017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a8686-423a-42ea-aaef-57c31b10a457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.956017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a8686-423a-42ea-aaef-57c31b10a457 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.956364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a8686-423a-42ea-aaef-57c31b10a457 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:05.956364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a8686-423a-42ea-aaef-57c31b10a457 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.956364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a8686-423a-42ea-aaef-57c31b10a457 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.963262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac9716-da5b-4446-ac74-7fdce7b8c833 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.968084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49503459-fd63-4d10-81c3-3cd6270376cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.968714 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49503459-fd63-4d10-81c3-3cd6270376cc admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.968988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49503459-fd63-4d10-81c3-3cd6270376cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.969289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49503459-fd63-4d10-81c3-3cd6270376cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.969412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 775/1566] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:05.969925 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49503459-fd63-4d10-81c3-3cd6270376cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.970444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a8686-423a-42ea-aaef-57c31b10a457 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:05.971575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49503459-fd63-4d10-81c3-3cd6270376cc admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.982669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a8686-423a-42ea-aaef-57c31b10a457 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:05.987611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807526b-42f8-4c6f-80d9-6115afe4f1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:05.988235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807526b-42f8-4c6f-80d9-6115afe4f1ce admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:05.988511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807526b-42f8-4c6f-80d9-6115afe4f1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:05.988779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807526b-42f8-4c6f-80d9-6115afe4f1ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:05.989926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807526b-42f8-4c6f-80d9-6115afe4f1ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:05.990226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807526b-42f8-4c6f-80d9-6115afe4f1ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:05.993974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 792/1567] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:05.994139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807526b-42f8-4c6f-80d9-6115afe4f1ce admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:05.994259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49503459-fd63-4d10-81c3-3cd6270376cc admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.004421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49503459-fd63-4d10-81c3-3cd6270376cc admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.009055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 776/1568] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:06.009055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807526b-42f8-4c6f-80d9-6115afe4f1ce admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.010788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70378f7f-fd5d-455e-b041-368063725e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.011414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70378f7f-fd5d-455e-b041-368063725e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.011732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70378f7f-fd5d-455e-b041-368063725e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.012005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70378f7f-fd5d-455e-b041-368063725e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.012658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70378f7f-fd5d-455e-b041-368063725e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.014666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70378f7f-fd5d-455e-b041-368063725e5d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.017759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807526b-42f8-4c6f-80d9-6115afe4f1ce admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.023337 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fa200-4383-461b-815c-6a05860fd17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.024169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fa200-4383-461b-815c-6a05860fd17b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.025795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fa200-4383-461b-815c-6a05860fd17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.025795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fa200-4383-461b-815c-6a05860fd17b admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.026014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:06.026492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fa200-4383-461b-815c-6a05860fd17b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:06.026492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fa200-4383-461b-815c-6a05860fd17b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.027240 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fa200-4383-461b-815c-6a05860fd17b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.040714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 777/1569] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:06.043202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fa200-4383-461b-815c-6a05860fd17b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.055302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fa200-4383-461b-815c-6a05860fd17b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.060104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec555875-4bfc-4436-90c0-8ea60c3078b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.060712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec555875-4bfc-4436-90c0-8ea60c3078b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.060995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec555875-4bfc-4436-90c0-8ea60c3078b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.061362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec555875-4bfc-4436-90c0-8ea60c3078b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.062108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec555875-4bfc-4436-90c0-8ea60c3078b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:06.062565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec555875-4bfc-4436-90c0-8ea60c3078b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.064241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec555875-4bfc-4436-90c0-8ea60c3078b1 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.083583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 778/1570] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:06.083583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec555875-4bfc-4436-90c0-8ea60c3078b1 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.089797 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec555875-4bfc-4436-90c0-8ea60c3078b1 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.095206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d20201-1e48-4e42-a1f0-fb5a05890453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.095861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d20201-1e48-4e42-a1f0-fb5a05890453 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.096142 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d20201-1e48-4e42-a1f0-fb5a05890453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.096421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d20201-1e48-4e42-a1f0-fb5a05890453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.097137 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d20201-1e48-4e42-a1f0-fb5a05890453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.099069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d20201-1e48-4e42-a1f0-fb5a05890453 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.110149 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 779/1571] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:06.110969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d20201-1e48-4e42-a1f0-fb5a05890453 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.121564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d20201-1e48-4e42-a1f0-fb5a05890453 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.126303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a933c-03e7-4a90-ba0c-2ba85e8b4d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.126905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a933c-03e7-4a90-ba0c-2ba85e8b4d87 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.127187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a933c-03e7-4a90-ba0c-2ba85e8b4d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.127468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a933c-03e7-4a90-ba0c-2ba85e8b4d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.128142 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a933c-03e7-4a90-ba0c-2ba85e8b4d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.130713 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a933c-03e7-4a90-ba0c-2ba85e8b4d87 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.146230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 793/1572] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:05 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:06.150131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70378f7f-fd5d-455e-b041-368063725e5d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.213042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70378f7f-fd5d-455e-b041-368063725e5d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.217694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e6e1d-d8e3-498d-bb80-9cfc912c1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.218306 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e6e1d-d8e3-498d-bb80-9cfc912c1a70 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.218580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e6e1d-d8e3-498d-bb80-9cfc912c1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.218850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e6e1d-d8e3-498d-bb80-9cfc912c1a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.219523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e6e1d-d8e3-498d-bb80-9cfc912c1a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.221825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e6e1d-d8e3-498d-bb80-9cfc912c1a70 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.232216 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 780/1573] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:06 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:06.233047 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a933c-03e7-4a90-ba0c-2ba85e8b4d87 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.296563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a933c-03e7-4a90-ba0c-2ba85e8b4d87 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.301157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c5ce4-8df9-456c-b471-692741cfbf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.302376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c5ce4-8df9-456c-b471-692741cfbf49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.302654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c5ce4-8df9-456c-b471-692741cfbf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.302922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c5ce4-8df9-456c-b471-692741cfbf49 admin admin] PATH_INFO: `/v3/projects/57cf0da56c8540d6a6e96425e44f8159/users/1e8efcd5322d4c8daf4eeae5afc755f3/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.318345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c5ce4-8df9-456c-b471-692741cfbf49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57cf0da56c8540d6a6e96425e44f8159, user_id=1e8efcd5322d4c8daf4eeae5afc755f3, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.319378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WARNING py.warnings [None req-e69c5ce4-8df9-456c-b471-692741cfbf49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:29:06.319378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: warnings.warn(msg) Sep 05 12:29:06.319378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: Sep 05 12:29:06.321509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c5ce4-8df9-456c-b471-692741cfbf49 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.329337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 794/1574] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:06 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:06.330898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e6e1d-d8e3-498d-bb80-9cfc912c1a70 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.393250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e6e1d-d8e3-498d-bb80-9cfc912c1a70 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.398087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ddee-5408-40b3-a97d-288219f13268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.399414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ddee-5408-40b3-a97d-288219f13268 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.399414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ddee-5408-40b3-a97d-288219f13268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.399414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ddee-5408-40b3-a97d-288219f13268 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.399933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470ddee-5408-40b3-a97d-288219f13268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.401963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470ddee-5408-40b3-a97d-288219f13268 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.402294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 781/1575] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:06 2022] PUT /identity/v3/projects/57cf0da56c8540d6a6e96425e44f8159/users/1e8efcd5322d4c8daf4eeae5afc755f3/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:06.403113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c5ce4-8df9-456c-b471-692741cfbf49 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.464478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:06.478607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c5ce4-8df9-456c-b471-692741cfbf49 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.483388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2aefc-5da7-440d-9074-c73ec2148063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.484027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2aefc-5da7-440d-9074-c73ec2148063 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.484306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2aefc-5da7-440d-9074-c73ec2148063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.484573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2aefc-5da7-440d-9074-c73ec2148063 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.485231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2aefc-5da7-440d-9074-c73ec2148063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.487292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2aefc-5da7-440d-9074-c73ec2148063 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.491567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 795/1576] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:06 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:06.491567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ddee-5408-40b3-a97d-288219f13268 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.556157 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ddee-5408-40b3-a97d-288219f13268 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.560757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108567b9-5ec3-4fb2-97a4-f9ef3ef4e3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.561372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108567b9-5ec3-4fb2-97a4-f9ef3ef4e3dd admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.561644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108567b9-5ec3-4fb2-97a4-f9ef3ef4e3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.561912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108567b9-5ec3-4fb2-97a4-f9ef3ef4e3dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.562546 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108567b9-5ec3-4fb2-97a4-f9ef3ef4e3dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.564480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108567b9-5ec3-4fb2-97a4-f9ef3ef4e3dd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.569194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:06.591864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 782/1577] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:06 2022] POST /identity/v3/projects => generated 393 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:06.594312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2aefc-5da7-440d-9074-c73ec2148063 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.656314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2aefc-5da7-440d-9074-c73ec2148063 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.661573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f4ef7-d797-46be-8484-d9eb8076a7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.662232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f4ef7-d797-46be-8484-d9eb8076a7d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.662514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f4ef7-d797-46be-8484-d9eb8076a7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.662790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f4ef7-d797-46be-8484-d9eb8076a7d9 admin admin] PATH_INFO: `/v3/projects/4f6830d2e75a47c8a757d4a66c6695ca/users/de2273220e824339bd13d968c75df26b/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.689619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 796/1578] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:06 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:06.689619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108567b9-5ec3-4fb2-97a4-f9ef3ef4e3dd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.689948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f4ef7-d797-46be-8484-d9eb8076a7d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f6830d2e75a47c8a757d4a66c6695ca, user_id=de2273220e824339bd13d968c75df26b, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.691642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f4ef7-d797-46be-8484-d9eb8076a7d9 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.750239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108567b9-5ec3-4fb2-97a4-f9ef3ef4e3dd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.754982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91a2cb-f3fc-4ec0-bc54-9d1a95aff480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.755626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91a2cb-f3fc-4ec0-bc54-9d1a95aff480 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.755907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91a2cb-f3fc-4ec0-bc54-9d1a95aff480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.756179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91a2cb-f3fc-4ec0-bc54-9d1a95aff480 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.756809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91a2cb-f3fc-4ec0-bc54-9d1a95aff480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.757814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-2e91a2cb-f3fc-4ec0-bc54-9d1a95aff480 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:29:06.757814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Sep 05 12:29:06.757814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: Sep 05 12:29:06.759026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91a2cb-f3fc-4ec0-bc54-9d1a95aff480 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.791849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 783/1579] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:06 2022] PUT /identity/v3/projects/4f6830d2e75a47c8a757d4a66c6695ca/users/de2273220e824339bd13d968c75df26b/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:06.792754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f4ef7-d797-46be-8484-d9eb8076a7d9 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.855137 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f4ef7-d797-46be-8484-d9eb8076a7d9 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.857373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 797/1580] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:06 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:06.858279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91a2cb-f3fc-4ec0-bc54-9d1a95aff480 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:06.863040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cef067-e03b-40fd-b348-0252c8fafd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.863688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cef067-e03b-40fd-b348-0252c8fafd0b admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.864011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cef067-e03b-40fd-b348-0252c8fafd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.864271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cef067-e03b-40fd-b348-0252c8fafd0b admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.864908 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cef067-e03b-40fd-b348-0252c8fafd0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.866853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cef067-e03b-40fd-b348-0252c8fafd0b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.868734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91a2cb-f3fc-4ec0-bc54-9d1a95aff480 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:06.873696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df358-6ebb-4145-956a-aa8c740f5a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:06.874311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df358-6ebb-4145-956a-aa8c740f5a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.874592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df358-6ebb-4145-956a-aa8c740f5a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.878997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df358-6ebb-4145-956a-aa8c740f5a9c admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.878997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524df358-6ebb-4145-956a-aa8c740f5a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:06.878997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524df358-6ebb-4145-956a-aa8c740f5a9c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:06.956721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 784/1581] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:06 2022] POST /identity/v3/users => generated 294 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:06.958496 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc758ec-87e5-4b33-8c3a-466646f570ae None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:06.958784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc758ec-87e5-4b33-8c3a-466646f570ae None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:06.959057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc758ec-87e5-4b33-8c3a-466646f570ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:06.989245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1bc758ec-87e5-4b33-8c3a-466646f570ae None None] MFA Rules not processed for user `1e8efcd5322d4c8daf4eeae5afc755f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:06.991753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1bc758ec-87e5-4b33-8c3a-466646f570ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:06.997575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 798/1582] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:06 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:06.998328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df358-6ebb-4145-956a-aa8c740f5a9c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.008393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df358-6ebb-4145-956a-aa8c740f5a9c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.013229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac627949-5e9f-4d17-b30c-930c941c27e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.021050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627949-5e9f-4d17-b30c-930c941c27e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.021050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627949-5e9f-4d17-b30c-930c941c27e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.021050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627949-5e9f-4d17-b30c-930c941c27e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.021050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac627949-5e9f-4d17-b30c-930c941c27e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.021050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac627949-5e9f-4d17-b30c-930c941c27e9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.030401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:07.061841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 785/1583] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:07.062680 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc758ec-87e5-4b33-8c3a-466646f570ae None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.074095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc758ec-87e5-4b33-8c3a-466646f570ae None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.078676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb0add-ca03-4242-8a9e-e6fe540fb985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.079282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb0add-ca03-4242-8a9e-e6fe540fb985 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.079595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb0add-ca03-4242-8a9e-e6fe540fb985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.079875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb0add-ca03-4242-8a9e-e6fe540fb985 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.080527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcb0add-ca03-4242-8a9e-e6fe540fb985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.082463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcb0add-ca03-4242-8a9e-e6fe540fb985 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.128753 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 799/1584] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:06 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:07.130195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac627949-5e9f-4d17-b30c-930c941c27e9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.140361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac627949-5e9f-4d17-b30c-930c941c27e9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.145195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc527351-bd79-4c6d-9410-76f3321abe8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.146354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc527351-bd79-4c6d-9410-76f3321abe8c admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.146629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc527351-bd79-4c6d-9410-76f3321abe8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.146901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc527351-bd79-4c6d-9410-76f3321abe8c admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.147521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc527351-bd79-4c6d-9410-76f3321abe8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.149513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc527351-bd79-4c6d-9410-76f3321abe8c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.202630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 786/1585] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:07 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:07.203935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a2a20-4782-4634-b9bd-611f17b1ca48 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.204217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a2a20-4782-4634-b9bd-611f17b1ca48 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.204491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a2a20-4782-4634-b9bd-611f17b1ca48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.224706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-496a2a20-4782-4634-b9bd-611f17b1ca48 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:07.226179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-496a2a20-4782-4634-b9bd-611f17b1ca48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:07.235667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 787/1586] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:07.236812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a2a20-4782-4634-b9bd-611f17b1ca48 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.255629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a2a20-4782-4634-b9bd-611f17b1ca48 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.260685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69069a5-eef5-4ce0-9415-acd69ca4e02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.261340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69069a5-eef5-4ce0-9415-acd69ca4e02f admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.261627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69069a5-eef5-4ce0-9415-acd69ca4e02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.261899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69069a5-eef5-4ce0-9415-acd69ca4e02f admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.262542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69069a5-eef5-4ce0-9415-acd69ca4e02f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.264274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69069a5-eef5-4ce0-9415-acd69ca4e02f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.264894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 800/1587] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:07 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:07.269153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc527351-bd79-4c6d-9410-76f3321abe8c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.279222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 788/1588] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.279222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69069a5-eef5-4ce0-9415-acd69ca4e02f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.279530 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc527351-bd79-4c6d-9410-76f3321abe8c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.283108 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be17d69-9bfb-4bf7-9fa6-999648931806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.283769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17d69-9bfb-4bf7-9fa6-999648931806 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.284272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17d69-9bfb-4bf7-9fa6-999648931806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.284483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17d69-9bfb-4bf7-9fa6-999648931806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.285167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be17d69-9bfb-4bf7-9fa6-999648931806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.286767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be17d69-9bfb-4bf7-9fa6-999648931806 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.291959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69069a5-eef5-4ce0-9415-acd69ca4e02f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.296609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66bcb4-821c-40ff-b6ef-bc0c4bd79093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.297332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66bcb4-821c-40ff-b6ef-bc0c4bd79093 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.297462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 801/1589] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.297631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66bcb4-821c-40ff-b6ef-bc0c4bd79093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.297903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66bcb4-821c-40ff-b6ef-bc0c4bd79093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.298354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be17d69-9bfb-4bf7-9fa6-999648931806 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.298544 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66bcb4-821c-40ff-b6ef-bc0c4bd79093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.300262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66bcb4-821c-40ff-b6ef-bc0c4bd79093 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.310645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be17d69-9bfb-4bf7-9fa6-999648931806 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.310978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 789/1590] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.311827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66bcb4-821c-40ff-b6ef-bc0c4bd79093 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.315605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562da38-b4cb-49bf-97c0-7ecac88be7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.316213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562da38-b4cb-49bf-97c0-7ecac88be7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.316514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562da38-b4cb-49bf-97c0-7ecac88be7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.316788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562da38-b4cb-49bf-97c0-7ecac88be7e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.317952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562da38-b4cb-49bf-97c0-7ecac88be7e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.319888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562da38-b4cb-49bf-97c0-7ecac88be7e9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.323758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66bcb4-821c-40ff-b6ef-bc0c4bd79093 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.328519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fae01-568f-4439-b008-d863b823282f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.329131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fae01-568f-4439-b008-d863b823282f admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.329409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fae01-568f-4439-b008-d863b823282f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.329679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fae01-568f-4439-b008-d863b823282f admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.330315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fae01-568f-4439-b008-d863b823282f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.332018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fae01-568f-4439-b008-d863b823282f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.343482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 790/1591] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.344319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fae01-568f-4439-b008-d863b823282f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.354103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fae01-568f-4439-b008-d863b823282f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.359737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e9d95-969c-4d63-baf3-3d7136fffd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.360410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9d95-969c-4d63-baf3-3d7136fffd6b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.360685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9d95-969c-4d63-baf3-3d7136fffd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.360956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9d95-969c-4d63-baf3-3d7136fffd6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.361578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e9d95-969c-4d63-baf3-3d7136fffd6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.363199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e9d95-969c-4d63-baf3-3d7136fffd6b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.373237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 791/1592] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.374106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e9d95-969c-4d63-baf3-3d7136fffd6b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.430352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 802/1593] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:07 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:07.430352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562da38-b4cb-49bf-97c0-7ecac88be7e9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.430647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e9d95-969c-4d63-baf3-3d7136fffd6b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.480281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:07.486377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623cb8c-87fc-4cff-9454-2aebfeb5a6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.487052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623cb8c-87fc-4cff-9454-2aebfeb5a6ce service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.487328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623cb8c-87fc-4cff-9454-2aebfeb5a6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.487631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623cb8c-87fc-4cff-9454-2aebfeb5a6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.494916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562da38-b4cb-49bf-97c0-7ecac88be7e9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.497958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623cb8c-87fc-4cff-9454-2aebfeb5a6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.499415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623cb8c-87fc-4cff-9454-2aebfeb5a6ce service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.499947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27948120-5e62-4c9b-9697-93fe203e2f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.500582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27948120-5e62-4c9b-9697-93fe203e2f29 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.500863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27948120-5e62-4c9b-9697-93fe203e2f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.501140 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27948120-5e62-4c9b-9697-93fe203e2f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.501797 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27948120-5e62-4c9b-9697-93fe203e2f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.503521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27948120-5e62-4c9b-9697-93fe203e2f29 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.511095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 792/1594] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:07.511780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623cb8c-87fc-4cff-9454-2aebfeb5a6ce service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.518352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 803/1595] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.518352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27948120-5e62-4c9b-9697-93fe203e2f29 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.521849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623cb8c-87fc-4cff-9454-2aebfeb5a6ce service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.526818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c83b3-0d24-4932-af00-f7ee6d098362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.527365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c83b3-0d24-4932-af00-f7ee6d098362 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.527957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c83b3-0d24-4932-af00-f7ee6d098362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.527957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c83b3-0d24-4932-af00-f7ee6d098362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.528561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c83b3-0d24-4932-af00-f7ee6d098362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.529093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27948120-5e62-4c9b-9697-93fe203e2f29 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.530732 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c83b3-0d24-4932-af00-f7ee6d098362 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.534833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5009119-8ff6-4b8e-886e-8c9bcf0ac616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.535508 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5009119-8ff6-4b8e-886e-8c9bcf0ac616 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.535824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5009119-8ff6-4b8e-886e-8c9bcf0ac616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.536099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5009119-8ff6-4b8e-886e-8c9bcf0ac616 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.537053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5009119-8ff6-4b8e-886e-8c9bcf0ac616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.538400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5009119-8ff6-4b8e-886e-8c9bcf0ac616 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.540955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 793/1596] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.541875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c83b3-0d24-4932-af00-f7ee6d098362 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.553904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 804/1597] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.554092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c83b3-0d24-4932-af00-f7ee6d098362 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.555042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5009119-8ff6-4b8e-886e-8c9bcf0ac616 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.559057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37de1a-f4fb-4671-8627-074fd825a38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.559700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37de1a-f4fb-4671-8627-074fd825a38b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.559977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37de1a-f4fb-4671-8627-074fd825a38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.560248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37de1a-f4fb-4671-8627-074fd825a38b admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.560963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37de1a-f4fb-4671-8627-074fd825a38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.562622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37de1a-f4fb-4671-8627-074fd825a38b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.566655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5009119-8ff6-4b8e-886e-8c9bcf0ac616 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.571935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94905a-d83b-4afd-8b95-fe356c5c51a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.572516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94905a-d83b-4afd-8b95-fe356c5c51a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.572794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94905a-d83b-4afd-8b95-fe356c5c51a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.573098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94905a-d83b-4afd-8b95-fe356c5c51a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.573771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94905a-d83b-4afd-8b95-fe356c5c51a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.573878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 794/1598] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.574764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37de1a-f4fb-4671-8627-074fd825a38b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.575345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94905a-d83b-4afd-8b95-fe356c5c51a1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.582564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:07.586450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37de1a-f4fb-4671-8627-074fd825a38b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.586978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 805/1599] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.587814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94905a-d83b-4afd-8b95-fe356c5c51a1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.593759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda504-b884-41da-bb9d-e9088e5abadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.594375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda504-b884-41da-bb9d-e9088e5abadb admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.594646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda504-b884-41da-bb9d-e9088e5abadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.594921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda504-b884-41da-bb9d-e9088e5abadb admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.595583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dda504-b884-41da-bb9d-e9088e5abadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.597188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dda504-b884-41da-bb9d-e9088e5abadb admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.599640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94905a-d83b-4afd-8b95-fe356c5c51a1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.604606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a6c76-e13a-4606-ba78-68c7b2fccb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.611993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a6c76-e13a-4606-ba78-68c7b2fccb45 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.611993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a6c76-e13a-4606-ba78-68c7b2fccb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.611993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a6c76-e13a-4606-ba78-68c7b2fccb45 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.611993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a6c76-e13a-4606-ba78-68c7b2fccb45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.611993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a6c76-e13a-4606-ba78-68c7b2fccb45 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.612700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 795/1600] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.612700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda504-b884-41da-bb9d-e9088e5abadb admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.624640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda504-b884-41da-bb9d-e9088e5abadb admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.630477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d7582-4b81-439b-a83c-fcad552a3f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.631836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d7582-4b81-439b-a83c-fcad552a3f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.632347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d7582-4b81-439b-a83c-fcad552a3f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.632913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d7582-4b81-439b-a83c-fcad552a3f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.633985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d7582-4b81-439b-a83c-fcad552a3f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.635848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d7582-4b81-439b-a83c-fcad552a3f42 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.646929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 796/1601] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:07.648124 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d7582-4b81-439b-a83c-fcad552a3f42 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.658096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d7582-4b81-439b-a83c-fcad552a3f42 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.663670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9bf2d-67fc-4d75-9b0b-4e68284a699c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.670600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9bf2d-67fc-4d75-9b0b-4e68284a699c admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.670600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9bf2d-67fc-4d75-9b0b-4e68284a699c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.670600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9bf2d-67fc-4d75-9b0b-4e68284a699c admin admin] PATH_INFO: `/v3/projects/2c59704f053a485ca0d4b0a16f675cc9/users/70a571ba86fd47dbb6900b215b50f92f/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.681114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9bf2d-67fc-4d75-9b0b-4e68284a699c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c59704f053a485ca0d4b0a16f675cc9, user_id=70a571ba86fd47dbb6900b215b50f92f, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.683896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9bf2d-67fc-4d75-9b0b-4e68284a699c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.694517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 806/1602] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:07 2022] POST /identity/v3/users => generated 308 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:07.695433 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a6c76-e13a-4606-ba78-68c7b2fccb45 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.706450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a6c76-e13a-4606-ba78-68c7b2fccb45 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.711358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504d325-66ae-4759-9ee9-a43bcf5076e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.712092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504d325-66ae-4759-9ee9-a43bcf5076e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.712439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504d325-66ae-4759-9ee9-a43bcf5076e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.712841 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504d325-66ae-4759-9ee9-a43bcf5076e6 admin admin] PATH_INFO: `/v3/projects/094bae935a1e4738b0c51d33c6d89e3a/users/9a61a18900c44bea89647fc9ccdaf110/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.729590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504d325-66ae-4759-9ee9-a43bcf5076e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=094bae935a1e4738b0c51d33c6d89e3a, user_id=9a61a18900c44bea89647fc9ccdaf110, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.732492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504d325-66ae-4759-9ee9-a43bcf5076e6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.780190 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 797/1603] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:07 2022] PUT /identity/v3/projects/2c59704f053a485ca0d4b0a16f675cc9/users/70a571ba86fd47dbb6900b215b50f92f/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:07.781293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9bf2d-67fc-4d75-9b0b-4e68284a699c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.814577 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 807/1604] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:07 2022] PUT /identity/v3/projects/094bae935a1e4738b0c51d33c6d89e3a/users/9a61a18900c44bea89647fc9ccdaf110/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:07.814577 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504d325-66ae-4759-9ee9-a43bcf5076e6 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:07.854442 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9bf2d-67fc-4d75-9b0b-4e68284a699c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.898294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504d325-66ae-4759-9ee9-a43bcf5076e6 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:07.904293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e8745-e88f-402f-9038-95c07d73daec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.904979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e8745-e88f-402f-9038-95c07d73daec admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.905257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e8745-e88f-402f-9038-95c07d73daec admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.905534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e8745-e88f-402f-9038-95c07d73daec admin admin] PATH_INFO: `/v3/projects/19911e0c0b444897b621ce06dbd8296a/users/69f976789f894296a86b4cf651fdc3ae/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.924538 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e8745-e88f-402f-9038-95c07d73daec admin admin] RBAC: Authorizing `identity:create_grant(project_id=19911e0c0b444897b621ce06dbd8296a, user_id=69f976789f894296a86b4cf651fdc3ae, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.927646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5be0f4-7a7c-4918-9a0e-1366c64270ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:07.928197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e8745-e88f-402f-9038-95c07d73daec admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.928495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5be0f4-7a7c-4918-9a0e-1366c64270ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:07.928833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5be0f4-7a7c-4918-9a0e-1366c64270ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:07.929325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5be0f4-7a7c-4918-9a0e-1366c64270ab admin admin] PATH_INFO: `/v3/projects/28e5970aef734620b03cee214ad5dce1/users/ce65feb5777a4893af9d36d225894047/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:07.945853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5be0f4-7a7c-4918-9a0e-1366c64270ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e5970aef734620b03cee214ad5dce1, user_id=ce65feb5777a4893af9d36d225894047, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:07.947967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5be0f4-7a7c-4918-9a0e-1366c64270ab admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:07.996796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 808/1605] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:07 2022] PUT /identity/v3/projects/19911e0c0b444897b621ce06dbd8296a/users/69f976789f894296a86b4cf651fdc3ae/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:07.997601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e8745-e88f-402f-9038-95c07d73daec admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:08.035381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 798/1606] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:07 2022] PUT /identity/v3/projects/28e5970aef734620b03cee214ad5dce1/users/ce65feb5777a4893af9d36d225894047/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:08.035381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5be0f4-7a7c-4918-9a0e-1366c64270ab admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:08.069955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e8745-e88f-402f-9038-95c07d73daec admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:08.078516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:08.096394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:08.123992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5be0f4-7a7c-4918-9a0e-1366c64270ab admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:08.131062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b978bb-9036-4e41-86a8-8c18bfe00d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:08.133045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b978bb-9036-4e41-86a8-8c18bfe00d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.133045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b978bb-9036-4e41-86a8-8c18bfe00d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.133045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b978bb-9036-4e41-86a8-8c18bfe00d75 admin admin] PATH_INFO: `/v3/projects/bc9d09b0d78847e5af5665e83dc34be9/users/0c22e8434a5248338778ab9a6da59434/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.136865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4216b7-5073-4322-bbfc-d5b6d8d1fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:08.137581 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4216b7-5073-4322-bbfc-d5b6d8d1fd8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.138005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4216b7-5073-4322-bbfc-d5b6d8d1fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.138282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4216b7-5073-4322-bbfc-d5b6d8d1fd8e admin admin] PATH_INFO: `/v3/projects/442bf64c97d945feb79f64cdd66baa28/users/f5543fbd902d441cb6f03d3155f1af8d/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.154853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b978bb-9036-4e41-86a8-8c18bfe00d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc9d09b0d78847e5af5665e83dc34be9, user_id=0c22e8434a5248338778ab9a6da59434, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:08.162360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b978bb-9036-4e41-86a8-8c18bfe00d75 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:08.162532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4216b7-5073-4322-bbfc-d5b6d8d1fd8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=442bf64c97d945feb79f64cdd66baa28, user_id=f5543fbd902d441cb6f03d3155f1af8d, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:08.163603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4216b7-5073-4322-bbfc-d5b6d8d1fd8e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:08.236517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 809/1608] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:07 2022] PUT /identity/v3/projects/442bf64c97d945feb79f64cdd66baa28/users/f5543fbd902d441cb6f03d3155f1af8d/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:08.237882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4216b7-5073-4322-bbfc-d5b6d8d1fd8e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:08.249033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 799/1608] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:08 2022] PUT /identity/v3/projects/bc9d09b0d78847e5af5665e83dc34be9/users/0c22e8434a5248338778ab9a6da59434/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:08.250638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d0722-febf-4dec-801a-c0570126f456 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.251035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d0722-febf-4dec-801a-c0570126f456 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.251384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d0722-febf-4dec-801a-c0570126f456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.283382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-901d0722-febf-4dec-801a-c0570126f456 None None] MFA Rules not processed for user `70a571ba86fd47dbb6900b215b50f92f`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:08.285033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-901d0722-febf-4dec-801a-c0570126f456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:08.315064 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4216b7-5073-4322-bbfc-d5b6d8d1fd8e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:08.363715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 800/1609] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:08.366436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f05187-1ace-4432-84f2-a7baee7449de None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.366782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f05187-1ace-4432-84f2-a7baee7449de None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.367127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f05187-1ace-4432-84f2-a7baee7449de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.390404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c75c0c-2b47-4489-862d-98b95d336d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:08.396993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c75c0c-2b47-4489-862d-98b95d336d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.396993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c75c0c-2b47-4489-862d-98b95d336d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.396993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c75c0c-2b47-4489-862d-98b95d336d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.396993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c75c0c-2b47-4489-862d-98b95d336d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:08.396993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c75c0c-2b47-4489-862d-98b95d336d5f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:08.400870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-29f05187-1ace-4432-84f2-a7baee7449de None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:08.402150 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-29f05187-1ace-4432-84f2-a7baee7449de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:08.410059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 801/1610] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:08.411681 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 810/1611] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:08.411869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5106c6-2f3f-4db8-aac2-1eae5c066115 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.412777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5106c6-2f3f-4db8-aac2-1eae5c066115 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.413041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5106c6-2f3f-4db8-aac2-1eae5c066115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.420283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95783c0-5460-4e0c-9f8c-f47ae1815e03 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.420825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95783c0-5460-4e0c-9f8c-f47ae1815e03 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.420965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95783c0-5460-4e0c-9f8c-f47ae1815e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.433249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-9c5106c6-2f3f-4db8-aac2-1eae5c066115 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:08.434442 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9c5106c6-2f3f-4db8-aac2-1eae5c066115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:08.442193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 802/1612] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:08.443369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbaa75-44c6-4605-956c-9756c2e32df4 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.443729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbaa75-44c6-4605-956c-9756c2e32df4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.444010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbaa75-44c6-4605-956c-9756c2e32df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.460086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c95783c0-5460-4e0c-9f8c-f47ae1815e03 None None] MFA Rules not processed for user `ce65feb5777a4893af9d36d225894047`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:08.460086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c95783c0-5460-4e0c-9f8c-f47ae1815e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:08.473356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-63bbaa75-44c6-4605-956c-9756c2e32df4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:08.475718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-63bbaa75-44c6-4605-956c-9756c2e32df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:08.487493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 803/1613] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:08.488613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bfbea-cb16-4fd5-925f-e422ba2e4fe7 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.493120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bfbea-cb16-4fd5-925f-e422ba2e4fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.493120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bfbea-cb16-4fd5-925f-e422ba2e4fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.500783 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:08.525691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-452bfbea-cb16-4fd5-925f-e422ba2e4fe7 None None] MFA Rules not processed for user `0c22e8434a5248338778ab9a6da59434`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:08.526989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-452bfbea-cb16-4fd5-925f-e422ba2e4fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:08.554627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 811/1614] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:08.556528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472568ad-efeb-4cf9-88d8-4d31f4c3e912 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.556842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472568ad-efeb-4cf9-88d8-4d31f4c3e912 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.557088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472568ad-efeb-4cf9-88d8-4d31f4c3e912 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.558947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 812/1615] 10.222.0.62 () {54 vars in 962 bytes} [Mon Sep 5 12:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:29:08.560296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472568ad-efeb-4cf9-88d8-4d31f4c3e912 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:08.576362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472568ad-efeb-4cf9-88d8-4d31f4c3e912 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:08.581074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bc29a-8d70-44c9-a089-2d94e14647e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:08.584923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bc29a-8d70-44c9-a089-2d94e14647e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.584923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bc29a-8d70-44c9-a089-2d94e14647e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.584923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bc29a-8d70-44c9-a089-2d94e14647e0 admin admin] PATH_INFO: `/v3/projects/a9dc6b24620d4510bfc61bd2cab941e1/users/34d9488444e44bd9b717231184af5067/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.596941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 804/1616] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:08.597798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452bfbea-cb16-4fd5-925f-e422ba2e4fe7 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:08.599392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-452bfbea-cb16-4fd5-925f-e422ba2e4fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:08.604335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bc29a-8d70-44c9-a089-2d94e14647e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9dc6b24620d4510bfc61bd2cab941e1, user_id=34d9488444e44bd9b717231184af5067, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:08.607736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bc29a-8d70-44c9-a089-2d94e14647e0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:08.639410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:08.664714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452bfbea-cb16-4fd5-925f-e422ba2e4fe7 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:08.668482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581473f-1a20-462b-9e61-49de83ec8530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:08.669097 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581473f-1a20-462b-9e61-49de83ec8530 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.669381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581473f-1a20-462b-9e61-49de83ec8530 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.669835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581473f-1a20-462b-9e61-49de83ec8530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.680173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581473f-1a20-462b-9e61-49de83ec8530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:08.681666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581473f-1a20-462b-9e61-49de83ec8530 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:08.691702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 805/1617] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:08.693046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581473f-1a20-462b-9e61-49de83ec8530 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:08.701727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581473f-1a20-462b-9e61-49de83ec8530 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:08.705572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58307961-37ed-4434-9554-6ea2a8bce1b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:08.711306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58307961-37ed-4434-9554-6ea2a8bce1b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.711306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58307961-37ed-4434-9554-6ea2a8bce1b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.711306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58307961-37ed-4434-9554-6ea2a8bce1b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.716807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58307961-37ed-4434-9554-6ea2a8bce1b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:08.718302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58307961-37ed-4434-9554-6ea2a8bce1b7 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:08.728577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 806/1618] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:08.732739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 813/1619] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:08 2022] PUT /identity/v3/projects/a9dc6b24620d4510bfc61bd2cab941e1/users/34d9488444e44bd9b717231184af5067/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:08.732739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bc29a-8d70-44c9-a089-2d94e14647e0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:08.733043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58307961-37ed-4434-9554-6ea2a8bce1b7 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:08.742187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58307961-37ed-4434-9554-6ea2a8bce1b7 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:08.804104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bc29a-8d70-44c9-a089-2d94e14647e0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:08.810166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c9cdb6-d4b8-4f7d-b600-5f0b85bed53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:08.810809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9cdb6-d4b8-4f7d-b600-5f0b85bed53c admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.811203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9cdb6-d4b8-4f7d-b600-5f0b85bed53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.811527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9cdb6-d4b8-4f7d-b600-5f0b85bed53c admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.812768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c9cdb6-d4b8-4f7d-b600-5f0b85bed53c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:08.812911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09106b6c-f53d-456a-84a2-194aabcca9dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:08.813419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09106b6c-f53d-456a-84a2-194aabcca9dd service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.813828 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09106b6c-f53d-456a-84a2-194aabcca9dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.814156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09106b6c-f53d-456a-84a2-194aabcca9dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.814723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c9cdb6-d4b8-4f7d-b600-5f0b85bed53c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:08.825401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09106b6c-f53d-456a-84a2-194aabcca9dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:08.827401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09106b6c-f53d-456a-84a2-194aabcca9dd service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:08.840321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 807/1620] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:08.841841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cccc6-858e-4fd0-9533-20661583532c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.841989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cccc6-858e-4fd0-9533-20661583532c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.842223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cccc6-858e-4fd0-9533-20661583532c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.887934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d57cccc6-858e-4fd0-9533-20661583532c None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:08.889191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d57cccc6-858e-4fd0-9533-20661583532c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:08.934533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 814/1621] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:08 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:08.936258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0458d01-21f4-4aa9-9ed7-9a06fc5fd55c None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:08.936258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0458d01-21f4-4aa9-9ed7-9a06fc5fd55c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:08.936843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0458d01-21f4-4aa9-9ed7-9a06fc5fd55c None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:08.943003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 815/1622] 10.222.0.62 () {54 vars in 962 bytes} [Mon Sep 5 12:29:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:29:08.944220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0458d01-21f4-4aa9-9ed7-9a06fc5fd55c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:09.010149 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 808/1623] 10.222.0.62 () {58 vars in 1059 bytes} [Mon Sep 5 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:09.015893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cccc6-858e-4fd0-9533-20661583532c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:09.041155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0458d01-21f4-4aa9-9ed7-9a06fc5fd55c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:09.046638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8b7e7-4d7a-4650-826c-12500d36ba86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:09.047422 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8b7e7-4d7a-4650-826c-12500d36ba86 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:09.047885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8b7e7-4d7a-4650-826c-12500d36ba86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:09.048423 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8b7e7-4d7a-4650-826c-12500d36ba86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:09.049113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8b7e7-4d7a-4650-826c-12500d36ba86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:09.049235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cccc6-858e-4fd0-9533-20661583532c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:09.050921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8b7e7-4d7a-4650-826c-12500d36ba86 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:09.060454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763e8eb-c1ae-4376-8e5b-8fd829059b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:09.063371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763e8eb-c1ae-4376-8e5b-8fd829059b15 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:09.063371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763e8eb-c1ae-4376-8e5b-8fd829059b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:09.063768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763e8eb-c1ae-4376-8e5b-8fd829059b15 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:09.064809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763e8eb-c1ae-4376-8e5b-8fd829059b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:09.068940 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763e8eb-c1ae-4376-8e5b-8fd829059b15 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:09.069909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 816/1624] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:09.071412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8494f-2cc4-4ab0-a6f6-cc58b1d851b0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:09.071792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8494f-2cc4-4ab0-a6f6-cc58b1d851b0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:09.072131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8494f-2cc4-4ab0-a6f6-cc58b1d851b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:09.096474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1aa8494f-2cc4-4ab0-a6f6-cc58b1d851b0 None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:09.098407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1aa8494f-2cc4-4ab0-a6f6-cc58b1d851b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:09.176314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 809/1625] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:09 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:09.177745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763e8eb-c1ae-4376-8e5b-8fd829059b15 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:09.186611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 817/1626] 10.222.0.62 () {58 vars in 1059 bytes} [Mon Sep 5 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:09.187569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8494f-2cc4-4ab0-a6f6-cc58b1d851b0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:09.201371 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8494f-2cc4-4ab0-a6f6-cc58b1d851b0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:09.208317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183187e5-edcd-46f8-8921-6a40003c0d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:09.209080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183187e5-edcd-46f8-8921-6a40003c0d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:09.209427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183187e5-edcd-46f8-8921-6a40003c0d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:09.209806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183187e5-edcd-46f8-8921-6a40003c0d74 admin admin] PATH_INFO: `/v3/projects/a9dc6b24620d4510bfc61bd2cab941e1/users/34d9488444e44bd9b717231184af5067/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:09.211994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183187e5-edcd-46f8-8921-6a40003c0d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9dc6b24620d4510bfc61bd2cab941e1, user_id=34d9488444e44bd9b717231184af5067, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:09.212601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763e8eb-c1ae-4376-8e5b-8fd829059b15 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:09.217835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183187e5-edcd-46f8-8921-6a40003c0d74 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:09.230954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 818/1627] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:09 2022] PUT /identity/v3/projects/a9dc6b24620d4510bfc61bd2cab941e1/users/34d9488444e44bd9b717231184af5067/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:09.231785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183187e5-edcd-46f8-8921-6a40003c0d74 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:09.240933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3b1b4-9eb3-441a-a2b8-b514eee3d05f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:09.241585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3b1b4-9eb3-441a-a2b8-b514eee3d05f service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:09.245035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3b1b4-9eb3-441a-a2b8-b514eee3d05f service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:09.246686 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3b1b4-9eb3-441a-a2b8-b514eee3d05f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:09.356992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183187e5-edcd-46f8-8921-6a40003c0d74 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:09.380282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3b1b4-9eb3-441a-a2b8-b514eee3d05f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:09.381575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad967779-0151-4c00-bf5d-c95ff8d3c7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:09.382327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad967779-0151-4c00-bf5d-c95ff8d3c7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:09.382611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad967779-0151-4c00-bf5d-c95ff8d3c7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:09.384165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad967779-0151-4c00-bf5d-c95ff8d3c7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:09.384834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3b1b4-9eb3-441a-a2b8-b514eee3d05f service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:09.386272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad967779-0151-4c00-bf5d-c95ff8d3c7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:09.389383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad967779-0151-4c00-bf5d-c95ff8d3c7e5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:09.412629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 819/1628] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:09.413425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 810/1629] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:09.413681 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad967779-0151-4c00-bf5d-c95ff8d3c7e5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:09.417799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f26d3-22cd-4358-aee8-af6725bc2ebf None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:09.418867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f26d3-22cd-4358-aee8-af6725bc2ebf None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:09.426193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f26d3-22cd-4358-aee8-af6725bc2ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:09.467310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:09.494305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-291f26d3-22cd-4358-aee8-af6725bc2ebf None None] MFA Rules not processed for user `34d9488444e44bd9b717231184af5067`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:09.496546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-291f26d3-22cd-4358-aee8-af6725bc2ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:09.557598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad967779-0151-4c00-bf5d-c95ff8d3c7e5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:09.572413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd8e8c-53e7-44d2-8ed6-a56ce721613f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:09.573379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd8e8c-53e7-44d2-8ed6-a56ce721613f service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:09.573654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd8e8c-53e7-44d2-8ed6-a56ce721613f service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:09.573926 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd8e8c-53e7-44d2-8ed6-a56ce721613f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:09.619405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:09.663987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 811/1630] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:09.664840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f26d3-22cd-4358-aee8-af6725bc2ebf None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:09.701831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f26d3-22cd-4358-aee8-af6725bc2ebf None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:09.712963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e06ed-9a67-4bbf-86a6-f23d9f386755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:09.713642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e06ed-9a67-4bbf-86a6-f23d9f386755 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:09.715084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e06ed-9a67-4bbf-86a6-f23d9f386755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:09.715273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e06ed-9a67-4bbf-86a6-f23d9f386755 admin admin] PATH_INFO: `/v3/projects/8b8d923e5f2b4d3eaaa6e47f4186d0ae/users/f450a5a50f324781a78234cdeb6a5a87/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:09.736300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd8e8c-53e7-44d2-8ed6-a56ce721613f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:09.740180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd8e8c-53e7-44d2-8ed6-a56ce721613f service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:09.751833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e06ed-9a67-4bbf-86a6-f23d9f386755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b8d923e5f2b4d3eaaa6e47f4186d0ae, user_id=f450a5a50f324781a78234cdeb6a5a87, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:09.757576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e06ed-9a67-4bbf-86a6-f23d9f386755 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:09.764753 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 820/1631] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:09.766820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd8e8c-53e7-44d2-8ed6-a56ce721613f service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:09.849345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 812/1632] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:09 2022] PUT /identity/v3/projects/8b8d923e5f2b4d3eaaa6e47f4186d0ae/users/f450a5a50f324781a78234cdeb6a5a87/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:09.850072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e06ed-9a67-4bbf-86a6-f23d9f386755 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:09.867693 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd8e8c-53e7-44d2-8ed6-a56ce721613f service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:09.994451 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690e026-cf1d-4fd8-8c06-cc11e17ada08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:09.996280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690e026-cf1d-4fd8-8c06-cc11e17ada08 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.001296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690e026-cf1d-4fd8-8c06-cc11e17ada08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.001296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690e026-cf1d-4fd8-8c06-cc11e17ada08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.027117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e06ed-9a67-4bbf-86a6-f23d9f386755 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:10.039206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ee205-bdd5-4e29-b154-de8685a78d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:10.039851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ee205-bdd5-4e29-b154-de8685a78d26 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.040130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ee205-bdd5-4e29-b154-de8685a78d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.040403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ee205-bdd5-4e29-b154-de8685a78d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.041045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ee205-bdd5-4e29-b154-de8685a78d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:10.042663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ee205-bdd5-4e29-b154-de8685a78d26 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:10.076486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 813/1633] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:10.078211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ee205-bdd5-4e29-b154-de8685a78d26 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:10.101553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ee205-bdd5-4e29-b154-de8685a78d26 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:10.110542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12af37-6789-48be-9199-8ebb7bcd4032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:10.110542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12af37-6789-48be-9199-8ebb7bcd4032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.110542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12af37-6789-48be-9199-8ebb7bcd4032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.110542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12af37-6789-48be-9199-8ebb7bcd4032 admin admin] PATH_INFO: `/v3/projects/8b8d923e5f2b4d3eaaa6e47f4186d0ae/users/f450a5a50f324781a78234cdeb6a5a87/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.112950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f12af37-6789-48be-9199-8ebb7bcd4032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b8d923e5f2b4d3eaaa6e47f4186d0ae, user_id=f450a5a50f324781a78234cdeb6a5a87, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:10.116571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f12af37-6789-48be-9199-8ebb7bcd4032 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:10.119030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690e026-cf1d-4fd8-8c06-cc11e17ada08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:10.120599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690e026-cf1d-4fd8-8c06-cc11e17ada08 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:10.134425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 814/1634] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:10 2022] PUT /identity/v3/projects/8b8d923e5f2b4d3eaaa6e47f4186d0ae/users/f450a5a50f324781a78234cdeb6a5a87/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:10.136501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae30bd-5a23-4ee4-9e62-246f9e677624 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.141567 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae30bd-5a23-4ee4-9e62-246f9e677624 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.141959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae30bd-5a23-4ee4-9e62-246f9e677624 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.148213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 815/1635] 10.222.0.62 () {54 vars in 912 bytes} [Mon Sep 5 12:29:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:29:10.149809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8501a21-893e-4d02-b033-6e53a24f4728 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.151121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 821/1636] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:10.151478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8501a21-893e-4d02-b033-6e53a24f4728 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.151857 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8501a21-893e-4d02-b033-6e53a24f4728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.153381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fed5f-03d8-4692-953f-48833d9bd38b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.153955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fed5f-03d8-4692-953f-48833d9bd38b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.154526 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fed5f-03d8-4692-953f-48833d9bd38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.184959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-cc5fed5f-03d8-4692-953f-48833d9bd38b None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:10.186412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cc5fed5f-03d8-4692-953f-48833d9bd38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:10.227089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c8501a21-893e-4d02-b033-6e53a24f4728 None None] MFA Rules not processed for user `f450a5a50f324781a78234cdeb6a5a87`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:10.229801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c8501a21-893e-4d02-b033-6e53a24f4728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:10.301561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 822/1637] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:10.312102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f22e9b-41c2-45a5-8128-ac7b8623b061 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.312737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f22e9b-41c2-45a5-8128-ac7b8623b061 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.313244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f22e9b-41c2-45a5-8128-ac7b8623b061 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.315126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 823/1638] 10.222.0.62 () {54 vars in 912 bytes} [Mon Sep 5 12:29:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 12:29:10.317222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f22e9b-41c2-45a5-8128-ac7b8623b061 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:10.377283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 816/1639] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:10.377283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7ea30-49f4-49d2-be05-aa354ee7f02b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.377283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7ea30-49f4-49d2-be05-aa354ee7f02b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.377283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7ea30-49f4-49d2-be05-aa354ee7f02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.395912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f22e9b-41c2-45a5-8128-ac7b8623b061 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:10.406081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eafaa7-f5f7-44af-a51e-c1a301c16c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:10.408252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eafaa7-f5f7-44af-a51e-c1a301c16c51 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.409571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eafaa7-f5f7-44af-a51e-c1a301c16c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.410319 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eafaa7-f5f7-44af-a51e-c1a301c16c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.423438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7eb7ea30-49f4-49d2-be05-aa354ee7f02b None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:10.424841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7eb7ea30-49f4-49d2-be05-aa354ee7f02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:10.432118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eafaa7-f5f7-44af-a51e-c1a301c16c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:10.435524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eafaa7-f5f7-44af-a51e-c1a301c16c51 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:10.443031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 817/1640] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:10.444463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7ea30-49f4-49d2-be05-aa354ee7f02b None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:10.462719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7ea30-49f4-49d2-be05-aa354ee7f02b None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:10.466888 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f618d-6c38-4e49-8267-cb75c38524de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:10.467519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f618d-6c38-4e49-8267-cb75c38524de service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.467889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f618d-6c38-4e49-8267-cb75c38524de service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.468167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f618d-6c38-4e49-8267-cb75c38524de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.475393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 824/1641] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:10.480134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f618d-6c38-4e49-8267-cb75c38524de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:10.481715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f618d-6c38-4e49-8267-cb75c38524de service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:10.493826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 818/1642] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:10.791894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:10.791894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eafaa7-f5f7-44af-a51e-c1a301c16c51 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:10.791894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eafaa7-f5f7-44af-a51e-c1a301c16c51 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:10.791894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40734bf8-4436-423c-b1c9-47bb51623628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:10.798615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40734bf8-4436-423c-b1c9-47bb51623628 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:10.798615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40734bf8-4436-423c-b1c9-47bb51623628 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:10.798615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40734bf8-4436-423c-b1c9-47bb51623628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:10.798615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40734bf8-4436-423c-b1c9-47bb51623628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:10.798615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40734bf8-4436-423c-b1c9-47bb51623628 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:10.814180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 825/1643] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:11.310877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:11.941422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f618d-6c38-4e49-8267-cb75c38524de service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:12.151752 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f618d-6c38-4e49-8267-cb75c38524de service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:12.172276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:12.173110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:12.173533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:12.173949 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] PATH_INFO: `/v3/users/1e8efcd5322d4c8daf4eeae5afc755f3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:12.175872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e8efcd5322d4c8daf4eeae5afc755f3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:12.177596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: WARNING py.warnings [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:29:12.177596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: warnings.warn(msg) Sep 05 12:29:12.177596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: Sep 05 12:29:12.190213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:12.474822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e8efcd5322d4c8daf4eeae5afc755f3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:12.476773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e8efcd5322d4c8daf4eeae5afc755f3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:12.566663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e8efcd5322d4c8daf4eeae5afc755f3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:12.613924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:12.616659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e8efcd5322d4c8daf4eeae5afc755f3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:12.617471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e8efcd5322d4c8daf4eeae5afc755f3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:12.644367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 819/1644] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:11 2022] DELETE /identity/v3/users/1e8efcd5322d4c8daf4eeae5afc755f3 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:12.648736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d32726-dde5-4722-b877-f952acf35096 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:12.649020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d32726-dde5-4722-b877-f952acf35096 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:12.649294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d32726-dde5-4722-b877-f952acf35096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:12.675872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-17d32726-dde5-4722-b877-f952acf35096 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:12.677673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-17d32726-dde5-4722-b877-f952acf35096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:12.777890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 826/1645] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:12.812461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:12.815479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:12.908608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:12.984093 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e78d01-7223-45b4-8874-75ca544322ea admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:12.990568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa31f64-2aac-460b-aba7-a0687efcfd3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:12.994497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa31f64-2aac-460b-aba7-a0687efcfd3f service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:12.994497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa31f64-2aac-460b-aba7-a0687efcfd3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:12.994497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa31f64-2aac-460b-aba7-a0687efcfd3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:13.008187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa31f64-2aac-460b-aba7-a0687efcfd3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:13.015520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa31f64-2aac-460b-aba7-a0687efcfd3f service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:13.032296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 820/1646] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:13.239190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d32726-dde5-4722-b877-f952acf35096 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:13.241494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-17d32726-dde5-4722-b877-f952acf35096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:13.264317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d32726-dde5-4722-b877-f952acf35096 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:13.286701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:13.287682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:13.287973 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:13.289029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] PATH_INFO: `/v3/users/ce65feb5777a4893af9d36d225894047` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:13.290483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce65feb5777a4893af9d36d225894047)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:13.301318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:13.589067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce65feb5777a4893af9d36d225894047'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:13.592472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce65feb5777a4893af9d36d225894047'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:13.745717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce65feb5777a4893af9d36d225894047'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:13.770971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:13.801134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce65feb5777a4893af9d36d225894047'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:13.804436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce65feb5777a4893af9d36d225894047'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:13.826756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 827/1647] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:13 2022] DELETE /identity/v3/users/ce65feb5777a4893af9d36d225894047 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:13.831195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e557fc5-a63b-4083-9a69-3917366fafe0 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:13.831769 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e557fc5-a63b-4083-9a69-3917366fafe0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:13.831769 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e557fc5-a63b-4083-9a69-3917366fafe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:13.871694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-9e557fc5-a63b-4083-9a69-3917366fafe0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:13.872932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9e557fc5-a63b-4083-9a69-3917366fafe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:14.025627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 821/1648] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:14.048993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:14.052219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:14.267704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b4cb8-4406-422a-a1e7-bf4c341ae5a5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:14.283222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4615e-b40a-4fa9-ae84-0650062049d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:14.284584 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4615e-b40a-4fa9-ae84-0650062049d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:14.284868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4615e-b40a-4fa9-ae84-0650062049d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:14.285148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4615e-b40a-4fa9-ae84-0650062049d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:14.329203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4615e-b40a-4fa9-ae84-0650062049d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:14.330688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4615e-b40a-4fa9-ae84-0650062049d9 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:14.358226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 828/1649] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:15.101118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e557fc5-a63b-4083-9a69-3917366fafe0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:15.103437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9e557fc5-a63b-4083-9a69-3917366fafe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:15.119821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e557fc5-a63b-4083-9a69-3917366fafe0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:15.127658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:15.128633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:15.129079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:15.131999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] PATH_INFO: `/v3/users/70a571ba86fd47dbb6900b215b50f92f` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:15.131999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70a571ba86fd47dbb6900b215b50f92f)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:15.134034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:15.233313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4615e-b40a-4fa9-ae84-0650062049d9 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:15.235468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fae4615e-b40a-4fa9-ae84-0650062049d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:15.283986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4615e-b40a-4fa9-ae84-0650062049d9 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:15.304186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:15.305657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:15.306466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:15.307752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] PATH_INFO: `/v3/projects/57cf0da56c8540d6a6e96425e44f8159` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:15.310754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57cf0da56c8540d6a6e96425e44f8159)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:15.315170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: WARNING py.warnings [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 12:29:15.315170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: warnings.warn(msg) Sep 05 12:29:15.315170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: Sep 05 12:29:15.319494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:15.397675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70a571ba86fd47dbb6900b215b50f92f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:15.398742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70a571ba86fd47dbb6900b215b50f92f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:15.526209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70a571ba86fd47dbb6900b215b50f92f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:15.554793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57cf0da56c8540d6a6e96425e44f8159', 'request_id': 'req-096c3e6c-051c-4063-aa69-551514477387'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:15.562699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70a571ba86fd47dbb6900b215b50f92f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:15.564324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bd74691a-2b69-4f76-a3f4-7cec9c069b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70a571ba86fd47dbb6900b215b50f92f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:15.572033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57cf0da56c8540d6a6e96425e44f8159', 'request_id': 'req-096c3e6c-051c-4063-aa69-551514477387'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:15.613061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 822/1650] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:15 2022] DELETE /identity/v3/users/70a571ba86fd47dbb6900b215b50f92f => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:15.614829 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddda820-7825-46b3-9831-d0a1e4c926df None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:15.615508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddda820-7825-46b3-9831-d0a1e4c926df None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:15.616013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddda820-7825-46b3-9831-d0a1e4c926df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:15.640683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57cf0da56c8540d6a6e96425e44f8159', 'request_id': 'req-096c3e6c-051c-4063-aa69-551514477387'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:15.659890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:15.675318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-3ddda820-7825-46b3-9831-d0a1e4c926df None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:15.678154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3ddda820-7825-46b3-9831-d0a1e4c926df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:15.679737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] The token cache is being invalidate because project 57cf0da56c8540d6a6e96425e44f8159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:15.681808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:15.683229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:15.688533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 829/1651] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:15 2022] DELETE /identity/v3/projects/57cf0da56c8540d6a6e96425e44f8159 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:15.693943 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:15.696355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:15.871281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c3e6c-051c-4063-aa69-551514477387 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:15.882747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 823/1652] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:15.888852 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:15.889532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:15.889819 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:15.893173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] PATH_INFO: `/v3/users/34d9488444e44bd9b717231184af5067` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:15.895979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34d9488444e44bd9b717231184af5067)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:15.897941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:15.906211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddda820-7825-46b3-9831-d0a1e4c926df None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:15.912100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3ddda820-7825-46b3-9831-d0a1e4c926df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:15.928227 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:16.046600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddda820-7825-46b3-9831-d0a1e4c926df None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:16.051870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90332d13-0cd2-4659-b374-d5ad31cdfde9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:16.052488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90332d13-0cd2-4659-b374-d5ad31cdfde9 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:16.052767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90332d13-0cd2-4659-b374-d5ad31cdfde9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:16.053036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90332d13-0cd2-4659-b374-d5ad31cdfde9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:16.065598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90332d13-0cd2-4659-b374-d5ad31cdfde9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:16.067905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90332d13-0cd2-4659-b374-d5ad31cdfde9 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:16.079910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 824/1653] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:29:16.216954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34d9488444e44bd9b717231184af5067'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:16.219178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34d9488444e44bd9b717231184af5067'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:16.323856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34d9488444e44bd9b717231184af5067'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:16.359991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34d9488444e44bd9b717231184af5067'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:16.361296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34d9488444e44bd9b717231184af5067'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:16.407181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 830/1654] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:15 2022] DELETE /identity/v3/users/34d9488444e44bd9b717231184af5067 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:29:17.122793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90332d13-0cd2-4659-b374-d5ad31cdfde9 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:17.130641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-90332d13-0cd2-4659-b374-d5ad31cdfde9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:17.146790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90332d13-0cd2-4659-b374-d5ad31cdfde9 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:17.324242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:17.324734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:17.325015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:17.325291 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] PATH_INFO: `/v3/projects/28e5970aef734620b03cee214ad5dce1` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:17.331146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28e5970aef734620b03cee214ad5dce1)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:17.336283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:17.527978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28e5970aef734620b03cee214ad5dce1', 'request_id': 'req-e81fa649-f0ef-4038-a18a-ee9cb0969a47'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:17.532482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:17.546030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28e5970aef734620b03cee214ad5dce1', 'request_id': 'req-e81fa649-f0ef-4038-a18a-ee9cb0969a47'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:17.624400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28e5970aef734620b03cee214ad5dce1', 'request_id': 'req-e81fa649-f0ef-4038-a18a-ee9cb0969a47'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:17.650542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] The token cache is being invalidate because project 28e5970aef734620b03cee214ad5dce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:17.653377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:17.654610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:17.660236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 825/1655] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:17 2022] DELETE /identity/v3/projects/28e5970aef734620b03cee214ad5dce1 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:17.664907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:17.667013 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:17.689774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b9e38-5c21-4cb2-996e-08a77abfe6e8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:17.832175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:17.833107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:17.833529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:17.833970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] PATH_INFO: `/v3/users/f450a5a50f324781a78234cdeb6a5a87` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:17.839373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f450a5a50f324781a78234cdeb6a5a87)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:17.841667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:18.006276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:18.008246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:18.034725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fa649-f0ef-4038-a18a-ee9cb0969a47 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:18.054406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:18.055131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:18.055422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:18.056262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] PATH_INFO: `/v3/projects/2c59704f053a485ca0d4b0a16f675cc9` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:18.058398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c59704f053a485ca0d4b0a16f675cc9)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:18.063114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:18.116200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f450a5a50f324781a78234cdeb6a5a87'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:18.120474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f450a5a50f324781a78234cdeb6a5a87'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:18.190656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f450a5a50f324781a78234cdeb6a5a87'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:18.220057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f450a5a50f324781a78234cdeb6a5a87'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:18.222252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6cbaca7b-33bf-44be-94dd-3df23a07a2ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f450a5a50f324781a78234cdeb6a5a87'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:18.242667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:18.261405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 831/1656] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:17 2022] DELETE /identity/v3/users/f450a5a50f324781a78234cdeb6a5a87 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:18.313196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c59704f053a485ca0d4b0a16f675cc9', 'request_id': 'req-729251a4-9ae5-4f70-b181-987ec27ee8f0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:18.322252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c59704f053a485ca0d4b0a16f675cc9', 'request_id': 'req-729251a4-9ae5-4f70-b181-987ec27ee8f0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:18.397317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c59704f053a485ca0d4b0a16f675cc9', 'request_id': 'req-729251a4-9ae5-4f70-b181-987ec27ee8f0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:18.424598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] The token cache is being invalidate because project 2c59704f053a485ca0d4b0a16f675cc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:18.425215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:18.426837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-729251a4-9ae5-4f70-b181-987ec27ee8f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:18.432669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 826/1657] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:18 2022] DELETE /identity/v3/projects/2c59704f053a485ca0d4b0a16f675cc9 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:18.437572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d2226-3d46-4d82-bfb0-f3c036528a9c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:18.438001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d2226-3d46-4d82-bfb0-f3c036528a9c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:18.438408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d2226-3d46-4d82-bfb0-f3c036528a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:18.490905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-651d2226-3d46-4d82-bfb0-f3c036528a9c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:18.492553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-651d2226-3d46-4d82-bfb0-f3c036528a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:18.623007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 832/1658] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:18.631697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6b2a1-318f-4e1e-bf61-e24e286b269f None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:18.632380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6b2a1-318f-4e1e-bf61-e24e286b269f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:18.633544 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6b2a1-318f-4e1e-bf61-e24e286b269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:18.685199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-62e6b2a1-318f-4e1e-bf61-e24e286b269f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:18.691142 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-62e6b2a1-318f-4e1e-bf61-e24e286b269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:18.730640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 827/1659] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:18.734784 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d2226-3d46-4d82-bfb0-f3c036528a9c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:18.758035 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d2226-3d46-4d82-bfb0-f3c036528a9c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:18.763582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e12a6-6e79-4c07-b524-0c13d8e8dcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:18.766458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e12a6-6e79-4c07-b524-0c13d8e8dcf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:18.766921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e12a6-6e79-4c07-b524-0c13d8e8dcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:18.767346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e12a6-6e79-4c07-b524-0c13d8e8dcf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:18.771196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e12a6-6e79-4c07-b524-0c13d8e8dcf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:18.771196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e12a6-6e79-4c07-b524-0c13d8e8dcf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:18.775642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e12a6-6e79-4c07-b524-0c13d8e8dcf4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:18.816467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6b2a1-318f-4e1e-bf61-e24e286b269f None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:18.818858 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-62e6b2a1-318f-4e1e-bf61-e24e286b269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:18.822122 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 833/1660] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:18.827771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e12a6-6e79-4c07-b524-0c13d8e8dcf4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:18.846681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6b2a1-318f-4e1e-bf61-e24e286b269f None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:18.863770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:18.865019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:18.865253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:18.865535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] PATH_INFO: `/v3/projects/a9dc6b24620d4510bfc61bd2cab941e1` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:18.869317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9dc6b24620d4510bfc61bd2cab941e1)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:18.871253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e12a6-6e79-4c07-b524-0c13d8e8dcf4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:18.871375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:18.896754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc224-7fe5-407f-9d90-7ddfff328541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:18.897381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc224-7fe5-407f-9d90-7ddfff328541 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:18.902555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc224-7fe5-407f-9d90-7ddfff328541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:18.902840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc224-7fe5-407f-9d90-7ddfff328541 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:18.903617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bc224-7fe5-407f-9d90-7ddfff328541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:18.905598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bc224-7fe5-407f-9d90-7ddfff328541 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:19.096266 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 834/1661] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:18 2022] POST /identity/v3/projects => generated 387 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:19.100739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc224-7fe5-407f-9d90-7ddfff328541 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:19.120579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9dc6b24620d4510bfc61bd2cab941e1', 'request_id': 'req-655603ef-1e7c-4928-9ce9-31cf408abf16'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:19.135239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9dc6b24620d4510bfc61bd2cab941e1', 'request_id': 'req-655603ef-1e7c-4928-9ce9-31cf408abf16'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:19.211737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9dc6b24620d4510bfc61bd2cab941e1', 'request_id': 'req-655603ef-1e7c-4928-9ce9-31cf408abf16'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:19.240634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] The token cache is being invalidate because project a9dc6b24620d4510bfc61bd2cab941e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:19.240972 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:19.242999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-655603ef-1e7c-4928-9ce9-31cf408abf16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:19.250317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 828/1662] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:18 2022] DELETE /identity/v3/projects/a9dc6b24620d4510bfc61bd2cab941e1 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:19.251815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38647174-7795-4627-a854-228f2b4ca8c9 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:19.253406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38647174-7795-4627-a854-228f2b4ca8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:19.253694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38647174-7795-4627-a854-228f2b4ca8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:19.311564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-38647174-7795-4627-a854-228f2b4ca8c9 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:19.315001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-38647174-7795-4627-a854-228f2b4ca8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:19.366325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:19.372788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc224-7fe5-407f-9d90-7ddfff328541 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:19.393949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd6ca-76ab-4eb8-88c7-55cd3ca6cb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:19.394791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd6ca-76ab-4eb8-88c7-55cd3ca6cb22 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:19.395078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd6ca-76ab-4eb8-88c7-55cd3ca6cb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:19.395389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd6ca-76ab-4eb8-88c7-55cd3ca6cb22 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:19.396065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecd6ca-76ab-4eb8-88c7-55cd3ca6cb22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:19.401450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecd6ca-76ab-4eb8-88c7-55cd3ca6cb22 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:19.421270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 829/1663] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:19.423024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24cfc-0193-4c23-a2a4-5dc32e239678 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:19.423318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24cfc-0193-4c23-a2a4-5dc32e239678 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:19.423624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24cfc-0193-4c23-a2a4-5dc32e239678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:19.470018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d3a24cfc-0193-4c23-a2a4-5dc32e239678 None None] MFA Rules not processed for user `de2273220e824339bd13d968c75df26b`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:19.471386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d3a24cfc-0193-4c23-a2a4-5dc32e239678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:19.560449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 835/1664] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:19 2022] POST /identity/v3/users => generated 294 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:19.561907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d89357-d378-44bb-b0f4-e6d376e6b94c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:19.562187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d89357-d378-44bb-b0f4-e6d376e6b94c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:19.562465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d89357-d378-44bb-b0f4-e6d376e6b94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:19.605844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 830/1665] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:19.608649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24cfc-0193-4c23-a2a4-5dc32e239678 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:19.621116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d3a24cfc-0193-4c23-a2a4-5dc32e239678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:19.631631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-02d89357-d378-44bb-b0f4-e6d376e6b94c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:19.633534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-02d89357-d378-44bb-b0f4-e6d376e6b94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:19.638302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24cfc-0193-4c23-a2a4-5dc32e239678 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:19.648234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fcea2-dd63-4cf8-b494-21da49ca2023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:19.649575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fcea2-dd63-4cf8-b494-21da49ca2023 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:19.650029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fcea2-dd63-4cf8-b494-21da49ca2023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:19.652164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fcea2-dd63-4cf8-b494-21da49ca2023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:19.653626 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fcea2-dd63-4cf8-b494-21da49ca2023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:19.662024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fcea2-dd63-4cf8-b494-21da49ca2023 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:19.669790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 836/1666] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:19.670770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d89357-d378-44bb-b0f4-e6d376e6b94c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:19.680723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-02d89357-d378-44bb-b0f4-e6d376e6b94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:19.684889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 831/1667] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:19.686452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fcea2-dd63-4cf8-b494-21da49ca2023 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:19.712733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fcea2-dd63-4cf8-b494-21da49ca2023 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:19.725691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164dd60b-1b82-4038-b9a6-8b586a811e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:19.726328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dd60b-1b82-4038-b9a6-8b586a811e9d admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:19.726613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dd60b-1b82-4038-b9a6-8b586a811e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:19.726891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dd60b-1b82-4038-b9a6-8b586a811e9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:19.727563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164dd60b-1b82-4038-b9a6-8b586a811e9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:19.727881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164dd60b-1b82-4038-b9a6-8b586a811e9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:19.729518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164dd60b-1b82-4038-b9a6-8b586a811e9d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:19.763072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 832/1668] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:19.764251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164dd60b-1b82-4038-b9a6-8b586a811e9d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:19.790614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164dd60b-1b82-4038-b9a6-8b586a811e9d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:19.804434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3ecba-b685-413b-92c0-aad12703d63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:19.805139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3ecba-b685-413b-92c0-aad12703d63e admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:19.805421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3ecba-b685-413b-92c0-aad12703d63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:19.805700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3ecba-b685-413b-92c0-aad12703d63e admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:19.806362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3ecba-b685-413b-92c0-aad12703d63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:19.808061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3ecba-b685-413b-92c0-aad12703d63e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:19.813153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:19.829606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 833/1669] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:19.830868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3ecba-b685-413b-92c0-aad12703d63e admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:19.855438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3ecba-b685-413b-92c0-aad12703d63e admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:19.868429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d89357-d378-44bb-b0f4-e6d376e6b94c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:19.868760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cd51e-b749-453a-9b90-9efc0002e8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:19.869344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cd51e-b749-453a-9b90-9efc0002e8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:19.869629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cd51e-b749-453a-9b90-9efc0002e8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:19.869904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cd51e-b749-453a-9b90-9efc0002e8a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:19.870549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cd51e-b749-453a-9b90-9efc0002e8a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:19.871821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dd901-3105-48dd-9ff9-5599f5732f28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:19.872512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cd51e-b749-453a-9b90-9efc0002e8a8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:19.873188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dd901-3105-48dd-9ff9-5599f5732f28 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:19.873691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dd901-3105-48dd-9ff9-5599f5732f28 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:19.874271 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dd901-3105-48dd-9ff9-5599f5732f28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:19.906598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5dd901-3105-48dd-9ff9-5599f5732f28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:19.908940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5dd901-3105-48dd-9ff9-5599f5732f28 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:19.924820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 837/1670] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:19.926372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dd901-3105-48dd-9ff9-5599f5732f28 service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:19.946037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dd901-3105-48dd-9ff9-5599f5732f28 service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:19.954971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c39162-38b2-4b92-90c4-980052f19cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:19.955717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c39162-38b2-4b92-90c4-980052f19cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:19.955998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c39162-38b2-4b92-90c4-980052f19cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:19.956518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c39162-38b2-4b92-90c4-980052f19cb4 admin admin] PATH_INFO: `/v3/projects/b03b6414ccce40da8111fa7dba1b02c5/users/f4a8481413114913a41e9db7221812a1/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:20.003982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c39162-38b2-4b92-90c4-980052f19cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03b6414ccce40da8111fa7dba1b02c5, user_id=f4a8481413114913a41e9db7221812a1, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:20.009032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c39162-38b2-4b92-90c4-980052f19cb4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:20.066915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 834/1671] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:19 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:20.073107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cd51e-b749-453a-9b90-9efc0002e8a8 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:20.135492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 838/1672] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:19 2022] PUT /identity/v3/projects/b03b6414ccce40da8111fa7dba1b02c5/users/f4a8481413114913a41e9db7221812a1/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:20.150598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84045fa0-99df-4140-999f-7c3a6a8b7d67 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:20.150881 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84045fa0-99df-4140-999f-7c3a6a8b7d67 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:20.151173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84045fa0-99df-4140-999f-7c3a6a8b7d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:20.262588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-84045fa0-99df-4140-999f-7c3a6a8b7d67 None None] MFA Rules not processed for user `f4a8481413114913a41e9db7221812a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:20.267037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-84045fa0-99df-4140-999f-7c3a6a8b7d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:20.315169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cd51e-b749-453a-9b90-9efc0002e8a8 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:20.416825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fab2e5-11ad-40e6-b644-24e0dd2e6ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:20.417878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fab2e5-11ad-40e6-b644-24e0dd2e6ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:20.418232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fab2e5-11ad-40e6-b644-24e0dd2e6ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:20.418576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fab2e5-11ad-40e6-b644-24e0dd2e6ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:20.419288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fab2e5-11ad-40e6-b644-24e0dd2e6ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:20.422808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fab2e5-11ad-40e6-b644-24e0dd2e6ed6 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:20.450707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 839/1673] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:20.453184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84045fa0-99df-4140-999f-7c3a6a8b7d67 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:20.468563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84045fa0-99df-4140-999f-7c3a6a8b7d67 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:20.485361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718c322-65bd-415d-94cb-1924a3b00f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:20.486307 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718c322-65bd-415d-94cb-1924a3b00f5d admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:20.486868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718c322-65bd-415d-94cb-1924a3b00f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:20.487281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718c322-65bd-415d-94cb-1924a3b00f5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:20.488104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718c322-65bd-415d-94cb-1924a3b00f5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:20.491654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718c322-65bd-415d-94cb-1924a3b00f5d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:20.516112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 835/1674] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:20 2022] POST /identity/v3/users => generated 291 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:20.520589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fab2e5-11ad-40e6-b644-24e0dd2e6ed6 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:20.546504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fab2e5-11ad-40e6-b644-24e0dd2e6ed6 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:20.561931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a630b-1ec8-430b-bc30-99a9a67dbecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:20.564560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a630b-1ec8-430b-bc30-99a9a67dbecc admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:20.565462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a630b-1ec8-430b-bc30-99a9a67dbecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:20.565462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a630b-1ec8-430b-bc30-99a9a67dbecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:20.566086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a630b-1ec8-430b-bc30-99a9a67dbecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:20.568579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a630b-1ec8-430b-bc30-99a9a67dbecc admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:20.591624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 836/1675] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:29:20.594676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a630b-1ec8-430b-bc30-99a9a67dbecc admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:20.611127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a630b-1ec8-430b-bc30-99a9a67dbecc admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:20.616877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa77d5-618b-43f7-8755-df89e744bc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:20.619319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa77d5-618b-43f7-8755-df89e744bc38 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:20.619319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa77d5-618b-43f7-8755-df89e744bc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:20.619319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa77d5-618b-43f7-8755-df89e744bc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:20.619583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa77d5-618b-43f7-8755-df89e744bc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:20.621422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa77d5-618b-43f7-8755-df89e744bc38 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:20.636957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 837/1676] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:20.638626 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa77d5-618b-43f7-8755-df89e744bc38 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:20.640797 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1eaa77d5-618b-43f7-8755-df89e744bc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:20.653880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 840/1677] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:20 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:20.655576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718c322-65bd-415d-94cb-1924a3b00f5d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:20.659770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa77d5-618b-43f7-8755-df89e744bc38 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:20.705654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:20.771471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:20.775355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:20.776053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:20.776693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] PATH_INFO: `/v3/projects/8b8d923e5f2b4d3eaaa6e47f4186d0ae` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:20.779087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b8d923e5f2b4d3eaaa6e47f4186d0ae)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:20.781930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:20.852986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718c322-65bd-415d-94cb-1924a3b00f5d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:20.863365 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52830ce-355b-4f93-ae27-69d63f8828dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:20.865772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52830ce-355b-4f93-ae27-69d63f8828dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:20.866054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52830ce-355b-4f93-ae27-69d63f8828dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:20.867169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52830ce-355b-4f93-ae27-69d63f8828dc admin admin] PATH_INFO: `/v3/projects/da505fda26fa4cb0ac5f011abb170010/users/fa1d3fd47f7b4f80a707e847305dd080/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:20.902481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52830ce-355b-4f93-ae27-69d63f8828dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=da505fda26fa4cb0ac5f011abb170010, user_id=fa1d3fd47f7b4f80a707e847305dd080, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:20.910008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52830ce-355b-4f93-ae27-69d63f8828dc admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:21.014340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:21.019639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 841/1678] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:20 2022] PUT /identity/v3/projects/da505fda26fa4cb0ac5f011abb170010/users/fa1d3fd47f7b4f80a707e847305dd080/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:21.022482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52830ce-355b-4f93-ae27-69d63f8828dc admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:21.038322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b8d923e5f2b4d3eaaa6e47f4186d0ae', 'request_id': 'req-ab7453e4-4d79-400c-b448-585353e001cd'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:21.054122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b8d923e5f2b4d3eaaa6e47f4186d0ae', 'request_id': 'req-ab7453e4-4d79-400c-b448-585353e001cd'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:21.136186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b8d923e5f2b4d3eaaa6e47f4186d0ae', 'request_id': 'req-ab7453e4-4d79-400c-b448-585353e001cd'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:21.156902 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] The token cache is being invalidate because project 8b8d923e5f2b4d3eaaa6e47f4186d0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:21.159480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:21.160234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab7453e4-4d79-400c-b448-585353e001cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:21.163699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52830ce-355b-4f93-ae27-69d63f8828dc admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:21.165257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 838/1679] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:20 2022] DELETE /identity/v3/projects/8b8d923e5f2b4d3eaaa6e47f4186d0ae => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:21.171178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82a9c2-8428-49b5-9fba-33e464153c93 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:21.172209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82a9c2-8428-49b5-9fba-33e464153c93 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:21.174379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82a9c2-8428-49b5-9fba-33e464153c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:21.236496 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-2e82a9c2-8428-49b5-9fba-33e464153c93 None None] MFA Rules not processed for user `fa1d3fd47f7b4f80a707e847305dd080`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:21.238516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2e82a9c2-8428-49b5-9fba-33e464153c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:21.329072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674150f5-9de7-47f6-945f-da5019f7c192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:21.330173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674150f5-9de7-47f6-945f-da5019f7c192 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:21.330484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674150f5-9de7-47f6-945f-da5019f7c192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:21.331165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674150f5-9de7-47f6-945f-da5019f7c192 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:21.332496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674150f5-9de7-47f6-945f-da5019f7c192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:21.335249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674150f5-9de7-47f6-945f-da5019f7c192 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:21.355780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 839/1680] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:21.357965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3ec79-1b19-41d0-b69d-efa24c9d926a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:21.358452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3ec79-1b19-41d0-b69d-efa24c9d926a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:21.358910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3ec79-1b19-41d0-b69d-efa24c9d926a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:21.384736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e0f3ec79-1b19-41d0-b69d-efa24c9d926a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:21.386224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e0f3ec79-1b19-41d0-b69d-efa24c9d926a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:21.406668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 840/1681] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:21.406668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3ec79-1b19-41d0-b69d-efa24c9d926a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:21.406668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e0f3ec79-1b19-41d0-b69d-efa24c9d926a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:21.428637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3ec79-1b19-41d0-b69d-efa24c9d926a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:21.436007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 842/1682] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:21 2022] POST /identity/v3/users => generated 293 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:21.437717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24822d6c-30ce-4514-97d1-5a286edf1546 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:21.438214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24822d6c-30ce-4514-97d1-5a286edf1546 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:21.440075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24822d6c-30ce-4514-97d1-5a286edf1546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:21.440765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c34fd-56da-4432-9f62-b1c88b63d94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:21.441452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c34fd-56da-4432-9f62-b1c88b63d94e admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:21.441728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c34fd-56da-4432-9f62-b1c88b63d94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:21.441996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c34fd-56da-4432-9f62-b1c88b63d94e admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:21.443776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c34fd-56da-4432-9f62-b1c88b63d94e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:21.445702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c34fd-56da-4432-9f62-b1c88b63d94e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:21.519203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-24822d6c-30ce-4514-97d1-5a286edf1546 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:21.522084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-24822d6c-30ce-4514-97d1-5a286edf1546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:21.546053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 843/1683] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:21.557866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24822d6c-30ce-4514-97d1-5a286edf1546 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:21.557866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-24822d6c-30ce-4514-97d1-5a286edf1546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:21.585365 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24822d6c-30ce-4514-97d1-5a286edf1546 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:21.589751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 841/1684] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:21 2022] POST /identity/v3/projects => generated 381 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:21.591027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c34fd-56da-4432-9f62-b1c88b63d94e admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:21.594447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4216c-a80d-4011-9fc7-1d646cd62bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:21.595500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4216c-a80d-4011-9fc7-1d646cd62bab admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:21.595993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4216c-a80d-4011-9fc7-1d646cd62bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:21.596502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4216c-a80d-4011-9fc7-1d646cd62bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:21.597391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4216c-a80d-4011-9fc7-1d646cd62bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:21.600878 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4216c-a80d-4011-9fc7-1d646cd62bab admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:21.640588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 844/1685] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:21.640588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4216c-a80d-4011-9fc7-1d646cd62bab admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:21.655229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4216c-a80d-4011-9fc7-1d646cd62bab admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:21.778576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:21.810455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c34fd-56da-4432-9f62-b1c88b63d94e admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:21.824495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496e724-dbfa-4342-bffa-a7cd7e0c6b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:21.830938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496e724-dbfa-4342-bffa-a7cd7e0c6b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:21.832431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496e724-dbfa-4342-bffa-a7cd7e0c6b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:21.833467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496e724-dbfa-4342-bffa-a7cd7e0c6b2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:21.837670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496e724-dbfa-4342-bffa-a7cd7e0c6b2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:21.838676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496e724-dbfa-4342-bffa-a7cd7e0c6b2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:21.845168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496e724-dbfa-4342-bffa-a7cd7e0c6b2b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:21.873596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421fe74-985d-4c8f-9b89-aa1f9033c611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:21.877503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421fe74-985d-4c8f-9b89-aa1f9033c611 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:21.878647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421fe74-985d-4c8f-9b89-aa1f9033c611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:21.879486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421fe74-985d-4c8f-9b89-aa1f9033c611 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:21.883703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421fe74-985d-4c8f-9b89-aa1f9033c611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:21.898709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421fe74-985d-4c8f-9b89-aa1f9033c611 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:21.911578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 842/1686] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:21.915304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496e724-dbfa-4342-bffa-a7cd7e0c6b2b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:21.948624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496e724-dbfa-4342-bffa-a7cd7e0c6b2b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:21.972397 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd0176-ce2b-43b2-998d-a6924065fd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:21.973048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd0176-ce2b-43b2-998d-a6924065fd56 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:21.973611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd0176-ce2b-43b2-998d-a6924065fd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:21.974045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd0176-ce2b-43b2-998d-a6924065fd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:21.974709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd0176-ce2b-43b2-998d-a6924065fd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:21.976473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd0176-ce2b-43b2-998d-a6924065fd56 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:21.991627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 843/1687] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:21.992481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd0176-ce2b-43b2-998d-a6924065fd56 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:22.028231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 845/1688] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:21 2022] POST /identity/v3/users => generated 297 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:22.030539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421fe74-985d-4c8f-9b89-aa1f9033c611 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:22.035192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd0176-ce2b-43b2-998d-a6924065fd56 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:22.077232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421fe74-985d-4c8f-9b89-aa1f9033c611 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:22.078713 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d4123-ac92-4643-be90-7cebf228d167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:22.079363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d4123-ac92-4643-be90-7cebf228d167 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:22.079685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d4123-ac92-4643-be90-7cebf228d167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:22.079967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d4123-ac92-4643-be90-7cebf228d167 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:22.080659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d4123-ac92-4643-be90-7cebf228d167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:22.086651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d4123-ac92-4643-be90-7cebf228d167 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:22.091305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c973a-a398-4ce5-8545-e7c1c0e42cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:22.092037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c973a-a398-4ce5-8545-e7c1c0e42cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:22.092316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c973a-a398-4ce5-8545-e7c1c0e42cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:22.092590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c973a-a398-4ce5-8545-e7c1c0e42cf8 admin admin] PATH_INFO: `/v3/projects/4ead8ea0c6324200b8f5ca4389a168f3/users/81599ae092de4b9aa8cad5a818d52890/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:22.097511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:22.115206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c973a-a398-4ce5-8545-e7c1c0e42cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ead8ea0c6324200b8f5ca4389a168f3, user_id=81599ae092de4b9aa8cad5a818d52890, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:22.119321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c973a-a398-4ce5-8545-e7c1c0e42cf8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:22.235674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 846/1689] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:22 2022] PUT /identity/v3/projects/4ead8ea0c6324200b8f5ca4389a168f3/users/81599ae092de4b9aa8cad5a818d52890/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:22.256405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c973a-a398-4ce5-8545-e7c1c0e42cf8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:22.270976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 844/1690] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:21 2022] POST /identity/v3/projects => generated 357 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:22.276658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8237-a4f0-466f-bdb5-5d4e2ad56fff None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:22.277312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8237-a4f0-466f-bdb5-5d4e2ad56fff None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:22.279981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8237-a4f0-466f-bdb5-5d4e2ad56fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:22.357450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-3b5b8237-a4f0-466f-bdb5-5d4e2ad56fff None None] MFA Rules not processed for user `81599ae092de4b9aa8cad5a818d52890`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:22.358855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3b5b8237-a4f0-466f-bdb5-5d4e2ad56fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:22.450122 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c973a-a398-4ce5-8545-e7c1c0e42cf8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:22.463607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 845/1691] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:22.466196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8237-a4f0-466f-bdb5-5d4e2ad56fff None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:22.583167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8237-a4f0-466f-bdb5-5d4e2ad56fff None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:22.604178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3460d52-03c8-41e6-9fe1-42cfdfe9653b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:22.605120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3460d52-03c8-41e6-9fe1-42cfdfe9653b admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:22.606560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3460d52-03c8-41e6-9fe1-42cfdfe9653b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:22.607064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3460d52-03c8-41e6-9fe1-42cfdfe9653b admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:22.607935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3460d52-03c8-41e6-9fe1-42cfdfe9653b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:22.610165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3460d52-03c8-41e6-9fe1-42cfdfe9653b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:22.706355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444f066-42ec-4087-892e-f262a33cfabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:22.707498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444f066-42ec-4087-892e-f262a33cfabe admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:22.708429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444f066-42ec-4087-892e-f262a33cfabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:22.709155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444f066-42ec-4087-892e-f262a33cfabe admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:22.709913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444f066-42ec-4087-892e-f262a33cfabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:22.716671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444f066-42ec-4087-892e-f262a33cfabe admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:22.741352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 846/1692] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:22 2022] POST /identity/v3/users => generated 279 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:22.742347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3460d52-03c8-41e6-9fe1-42cfdfe9653b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:22.791196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3460d52-03c8-41e6-9fe1-42cfdfe9653b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:22.803372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 847/1693] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 567 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:22.806437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444f066-42ec-4087-892e-f262a33cfabe admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:22.823788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16788ece-1678-49c1-aa00-90a93a998281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:22.825163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16788ece-1678-49c1-aa00-90a93a998281 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:22.826587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16788ece-1678-49c1-aa00-90a93a998281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:22.827405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16788ece-1678-49c1-aa00-90a93a998281 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:22.831089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16788ece-1678-49c1-aa00-90a93a998281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:22.838173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16788ece-1678-49c1-aa00-90a93a998281 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:22.843588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444f066-42ec-4087-892e-f262a33cfabe admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:22.869263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2569959-ce47-44d2-bd9f-cfbc2108545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:22.876833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569959-ce47-44d2-bd9f-cfbc2108545e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:22.877440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569959-ce47-44d2-bd9f-cfbc2108545e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:22.877764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569959-ce47-44d2-bd9f-cfbc2108545e admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:22.880029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2569959-ce47-44d2-bd9f-cfbc2108545e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:22.883741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2569959-ce47-44d2-bd9f-cfbc2108545e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:22.922767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 848/1694] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:22.923813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2569959-ce47-44d2-bd9f-cfbc2108545e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:22.958917 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2569959-ce47-44d2-bd9f-cfbc2108545e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:22.966404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6ae49-81fc-4e4d-bfa0-fc865c7dcb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:22.967318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6ae49-81fc-4e4d-bfa0-fc865c7dcb7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:22.967791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6ae49-81fc-4e4d-bfa0-fc865c7dcb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:22.968955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6ae49-81fc-4e4d-bfa0-fc865c7dcb7d admin admin] PATH_INFO: `/v3/projects/c604eae10f7d41babf230918ef27c585/users/bf182d5cf73844cfb3fe48d72480f11f/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:22.989665 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 847/1695] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:22 2022] POST /identity/v3/projects => generated 385 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:22.994102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16788ece-1678-49c1-aa00-90a93a998281 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:23.052492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6ae49-81fc-4e4d-bfa0-fc865c7dcb7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c604eae10f7d41babf230918ef27c585, user_id=bf182d5cf73844cfb3fe48d72480f11f, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:23.055151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6ae49-81fc-4e4d-bfa0-fc865c7dcb7d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:23.175956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 849/1696] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:22 2022] PUT /identity/v3/projects/c604eae10f7d41babf230918ef27c585/users/bf182d5cf73844cfb3fe48d72480f11f/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:23.175956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6ae49-81fc-4e4d-bfa0-fc865c7dcb7d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:23.243271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16788ece-1678-49c1-aa00-90a93a998281 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:23.384656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:23.425054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6ae49-81fc-4e4d-bfa0-fc865c7dcb7d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:23.446501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a0303-20e1-45c2-a819-192b0c58c2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:23.446687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:23.448231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a0303-20e1-45c2-a819-192b0c58c2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:23.448231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a0303-20e1-45c2-a819-192b0c58c2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:23.448231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a0303-20e1-45c2-a819-192b0c58c2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:23.448231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a0303-20e1-45c2-a819-192b0c58c2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:23.449859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a0303-20e1-45c2-a819-192b0c58c2f1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:23.497534 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf44d06-7b2d-4c57-b24f-60261b8dbe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:23.498226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf44d06-7b2d-4c57-b24f-60261b8dbe1e admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:23.498494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf44d06-7b2d-4c57-b24f-60261b8dbe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:23.499782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf44d06-7b2d-4c57-b24f-60261b8dbe1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:23.500579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf44d06-7b2d-4c57-b24f-60261b8dbe1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:23.510633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf44d06-7b2d-4c57-b24f-60261b8dbe1e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:23.539264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 848/1697] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 545 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:23.540349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf44d06-7b2d-4c57-b24f-60261b8dbe1e admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:23.576090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf44d06-7b2d-4c57-b24f-60261b8dbe1e admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:23.589415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7c5b4-5293-4601-8b76-6ae3232a618a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:23.590071 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c5b4-5293-4601-8b76-6ae3232a618a admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:23.590348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c5b4-5293-4601-8b76-6ae3232a618a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:23.590622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c5b4-5293-4601-8b76-6ae3232a618a admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:23.591274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7c5b4-5293-4601-8b76-6ae3232a618a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:23.599910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7c5b4-5293-4601-8b76-6ae3232a618a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:23.612836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 850/1698] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:23 2022] POST /identity/v3/users => generated 299 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:23.616563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a0303-20e1-45c2-a819-192b0c58c2f1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:23.626130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 849/1699] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:23.626130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7c5b4-5293-4601-8b76-6ae3232a618a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:23.630576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6bc7c5b4-5293-4601-8b76-6ae3232a618a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:23.650570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a0303-20e1-45c2-a819-192b0c58c2f1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:23.663661 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7c5b4-5293-4601-8b76-6ae3232a618a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:23.670827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba159ed-e4ea-4fc4-bd2e-0ec88c56e395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:23.671727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba159ed-e4ea-4fc4-bd2e-0ec88c56e395 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:23.672003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba159ed-e4ea-4fc4-bd2e-0ec88c56e395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:23.672271 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba159ed-e4ea-4fc4-bd2e-0ec88c56e395 admin admin] PATH_INFO: `/v3/projects/3504eb93c0ce4e40a219359a85f230fc/users/71f94e660c1d4e5ab35f8a1b7d78a84e/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:23.682313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:23.682313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:23.682313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:23.682313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] PATH_INFO: `/v3/users/0c22e8434a5248338778ab9a6da59434` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:23.685566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c22e8434a5248338778ab9a6da59434)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:23.687853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:23.712779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba159ed-e4ea-4fc4-bd2e-0ec88c56e395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3504eb93c0ce4e40a219359a85f230fc, user_id=71f94e660c1d4e5ab35f8a1b7d78a84e, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:23.717834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba159ed-e4ea-4fc4-bd2e-0ec88c56e395 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:23.844824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 851/1700] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:23 2022] PUT /identity/v3/projects/3504eb93c0ce4e40a219359a85f230fc/users/71f94e660c1d4e5ab35f8a1b7d78a84e/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:23.845903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba159ed-e4ea-4fc4-bd2e-0ec88c56e395 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:23.982215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c22e8434a5248338778ab9a6da59434'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:23.983269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c22e8434a5248338778ab9a6da59434'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:24.004306 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba159ed-e4ea-4fc4-bd2e-0ec88c56e395 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:24.020032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f64cb-7d84-4972-bbda-a2314d524dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:24.020955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f64cb-7d84-4972-bbda-a2314d524dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:24.021230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f64cb-7d84-4972-bbda-a2314d524dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:24.021682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f64cb-7d84-4972-bbda-a2314d524dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:24.022345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f64cb-7d84-4972-bbda-a2314d524dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:24.038558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f64cb-7d84-4972-bbda-a2314d524dbd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:24.064591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 852/1701] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:24.065818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f64cb-7d84-4972-bbda-a2314d524dbd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:24.071377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-800f64cb-7d84-4972-bbda-a2314d524dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:24.085585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c22e8434a5248338778ab9a6da59434'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:24.103019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f64cb-7d84-4972-bbda-a2314d524dbd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:24.118080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c22e8434a5248338778ab9a6da59434'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:24.119058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c22e8434a5248338778ab9a6da59434'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:24.119796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272325e8-f647-445c-af4d-1e6e0272e71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:24.122115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272325e8-f647-445c-af4d-1e6e0272e71e admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:24.122115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272325e8-f647-445c-af4d-1e6e0272e71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:24.122115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272325e8-f647-445c-af4d-1e6e0272e71e admin admin] PATH_INFO: `/v3/projects/c604eae10f7d41babf230918ef27c585/users/bf182d5cf73844cfb3fe48d72480f11f/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:24.127229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272325e8-f647-445c-af4d-1e6e0272e71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c604eae10f7d41babf230918ef27c585, user_id=bf182d5cf73844cfb3fe48d72480f11f, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:24.130250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272325e8-f647-445c-af4d-1e6e0272e71e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:24.166807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 853/1702] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:24 2022] PUT /identity/v3/projects/c604eae10f7d41babf230918ef27c585/users/bf182d5cf73844cfb3fe48d72480f11f/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:24.169465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ce6a-bd19-4c97-aeb3-5bb3fd426368 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:24.169757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ce6a-bd19-4c97-aeb3-5bb3fd426368 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:24.170039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ce6a-bd19-4c97-aeb3-5bb3fd426368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:24.175653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 850/1703] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:23 2022] DELETE /identity/v3/users/0c22e8434a5248338778ab9a6da59434 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:24.176350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:24.179895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:24.235789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-95d7ce6a-bd19-4c97-aeb3-5bb3fd426368 None None] MFA Rules not processed for user `71f94e660c1d4e5ab35f8a1b7d78a84e`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:24.236864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-95d7ce6a-bd19-4c97-aeb3-5bb3fd426368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:24.402854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 854/1704] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:24.404317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52471-ab8b-4e76-bef0-9325812611d5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:24.404818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52471-ab8b-4e76-bef0-9325812611d5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:24.406023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52471-ab8b-4e76-bef0-9325812611d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:24.442831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa6f0d-1383-48a1-8336-c16cf93bd792 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:24.475839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ff7e0-baf3-4811-87b6-879664115d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:24.477095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-eac52471-ab8b-4e76-bef0-9325812611d5 None None] MFA Rules not processed for user `bf182d5cf73844cfb3fe48d72480f11f`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:24.477608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ff7e0-baf3-4811-87b6-879664115d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:24.478033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ff7e0-baf3-4811-87b6-879664115d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:24.478349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-eac52471-ab8b-4e76-bef0-9325812611d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:24.478830 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ff7e0-baf3-4811-87b6-879664115d4b admin admin] PATH_INFO: `/v3/projects/63afe0aff49d4936ae83a76c56ead467/users/c2d9139857024eaab3778469c5c9a708/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:24.509951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ff7e0-baf3-4811-87b6-879664115d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=63afe0aff49d4936ae83a76c56ead467, user_id=c2d9139857024eaab3778469c5c9a708, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:24.518713 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ff7e0-baf3-4811-87b6-879664115d4b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:24.524690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:24.547280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:24.606127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 851/1705] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:24 2022] PUT /identity/v3/projects/63afe0aff49d4936ae83a76c56ead467/users/c2d9139857024eaab3778469c5c9a708/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:24.608305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd279992-4999-456d-a9d6-8857699481a4 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:24.608737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd279992-4999-456d-a9d6-8857699481a4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:24.609387 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd279992-4999-456d-a9d6-8857699481a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:24.716593 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-fd279992-4999-456d-a9d6-8857699481a4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:24.717983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fd279992-4999-456d-a9d6-8857699481a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:24.719439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 855/1706] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:24.720435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac52471-ab8b-4e76-bef0-9325812611d5 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:24.722024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-eac52471-ab8b-4e76-bef0-9325812611d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:24.748732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac52471-ab8b-4e76-bef0-9325812611d5 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:24.880931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28af71-a3ab-4e36-bcf5-13fc03777714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:24.881648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28af71-a3ab-4e36-bcf5-13fc03777714 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:24.881933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28af71-a3ab-4e36-bcf5-13fc03777714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:24.882248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28af71-a3ab-4e36-bcf5-13fc03777714 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:24.882973 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28af71-a3ab-4e36-bcf5-13fc03777714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:24.893465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28af71-a3ab-4e36-bcf5-13fc03777714 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:24.964415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 852/1707] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:24.966260 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd279992-4999-456d-a9d6-8857699481a4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:25.000836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd279992-4999-456d-a9d6-8857699481a4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:25.010450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb36a25-ee64-4d8c-8f09-13dc5354b14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:25.011096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb36a25-ee64-4d8c-8f09-13dc5354b14e admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:25.011378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb36a25-ee64-4d8c-8f09-13dc5354b14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:25.011699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb36a25-ee64-4d8c-8f09-13dc5354b14e admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:25.012341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb36a25-ee64-4d8c-8f09-13dc5354b14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:25.014012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb36a25-ee64-4d8c-8f09-13dc5354b14e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:25.033867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 856/1708] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:24 2022] POST /identity/v3/projects => generated 357 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:25.038387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28af71-a3ab-4e36-bcf5-13fc03777714 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:25.040788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8a28af71-a3ab-4e36-bcf5-13fc03777714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:25.042359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 853/1709] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:25.043208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb36a25-ee64-4d8c-8f09-13dc5354b14e admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:25.079513 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb36a25-ee64-4d8c-8f09-13dc5354b14e admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:25.088780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:25.147912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28af71-a3ab-4e36-bcf5-13fc03777714 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:25.153780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf28b7-3ce0-4fe1-9b38-e2508c2bb389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:25.154855 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf28b7-3ce0-4fe1-9b38-e2508c2bb389 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:25.155202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf28b7-3ce0-4fe1-9b38-e2508c2bb389 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:25.155755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf28b7-3ce0-4fe1-9b38-e2508c2bb389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:25.226447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:25.246833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf28b7-3ce0-4fe1-9b38-e2508c2bb389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:25.248994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf28b7-3ce0-4fe1-9b38-e2508c2bb389 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:25.270026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59decf6-df0f-41c4-9b53-96513306300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:25.270026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59decf6-df0f-41c4-9b53-96513306300f admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:25.270026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59decf6-df0f-41c4-9b53-96513306300f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:25.270482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 857/1710] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:25.270571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59decf6-df0f-41c4-9b53-96513306300f admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:25.270571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59decf6-df0f-41c4-9b53-96513306300f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:25.270731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf28b7-3ce0-4fe1-9b38-e2508c2bb389 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:25.270824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59decf6-df0f-41c4-9b53-96513306300f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:25.285416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf28b7-3ce0-4fe1-9b38-e2508c2bb389 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:25.294573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84580f94-7cee-4644-8454-60677bc13580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:25.295370 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84580f94-7cee-4644-8454-60677bc13580 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:25.295739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84580f94-7cee-4644-8454-60677bc13580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:25.298715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84580f94-7cee-4644-8454-60677bc13580 admin admin] PATH_INFO: `/v3/projects/63afe0aff49d4936ae83a76c56ead467/users/c2d9139857024eaab3778469c5c9a708/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:25.298715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84580f94-7cee-4644-8454-60677bc13580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63afe0aff49d4936ae83a76c56ead467, user_id=c2d9139857024eaab3778469c5c9a708, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:25.302393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84580f94-7cee-4644-8454-60677bc13580 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:25.335877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 858/1711] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:25 2022] PUT /identity/v3/projects/63afe0aff49d4936ae83a76c56ead467/users/c2d9139857024eaab3778469c5c9a708/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:25.337318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84580f94-7cee-4644-8454-60677bc13580 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:25.340257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-84580f94-7cee-4644-8454-60677bc13580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:25.401028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 854/1712] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:25 2022] POST /identity/v3/users => generated 285 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:25.401028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9cdbe-854d-46a1-83da-1a676e2098c5 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:25.401028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9cdbe-854d-46a1-83da-1a676e2098c5 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:25.402982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9cdbe-854d-46a1-83da-1a676e2098c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:25.518708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-a7a9cdbe-854d-46a1-83da-1a676e2098c5 None None] MFA Rules not processed for user `c2d9139857024eaab3778469c5c9a708`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:25.524674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a7a9cdbe-854d-46a1-83da-1a676e2098c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:25.540664 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84580f94-7cee-4644-8454-60677bc13580 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:25.549094 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762adf7c-ce31-4181-b7b3-0de533be7486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:25.550185 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762adf7c-ce31-4181-b7b3-0de533be7486 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:25.550495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762adf7c-ce31-4181-b7b3-0de533be7486 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:25.550771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762adf7c-ce31-4181-b7b3-0de533be7486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:25.672105 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 855/1713] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:25.673070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9cdbe-854d-46a1-83da-1a676e2098c5 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:25.713521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762adf7c-ce31-4181-b7b3-0de533be7486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:25.716156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762adf7c-ce31-4181-b7b3-0de533be7486 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:25.751186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 859/1714] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:25.752866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:25.762818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863eb38-f4b3-4a5c-8615-97f425605e0e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:25.762818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863eb38-f4b3-4a5c-8615-97f425605e0e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:25.762818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863eb38-f4b3-4a5c-8615-97f425605e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:25.849242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9cdbe-854d-46a1-83da-1a676e2098c5 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:25.870543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b470b3a-9849-458a-9801-fee8a19792a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:25.871571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b470b3a-9849-458a-9801-fee8a19792a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:25.872220 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b470b3a-9849-458a-9801-fee8a19792a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:25.872703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b470b3a-9849-458a-9801-fee8a19792a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:25.873589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b470b3a-9849-458a-9801-fee8a19792a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:25.875472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b470b3a-9849-458a-9801-fee8a19792a3 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:25.885337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9863eb38-f4b3-4a5c-8615-97f425605e0e None None] MFA Rules not processed for user `69f976789f894296a86b4cf651fdc3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:25.886340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9863eb38-f4b3-4a5c-8615-97f425605e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:25.906418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 856/1715] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:25.907649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b470b3a-9849-458a-9801-fee8a19792a3 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:26.088952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 860/1716] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:26.090372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151b3ee-8ab5-4a7d-b4e7-0f40afda75f5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:26.090656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151b3ee-8ab5-4a7d-b4e7-0f40afda75f5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:26.091080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151b3ee-8ab5-4a7d-b4e7-0f40afda75f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:26.096363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b470b3a-9849-458a-9801-fee8a19792a3 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:26.105751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e1c05-3fab-40cb-a2a3-f58fd42c7d57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:26.106392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e1c05-3fab-40cb-a2a3-f58fd42c7d57 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:26.106729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e1c05-3fab-40cb-a2a3-f58fd42c7d57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:26.106951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e1c05-3fab-40cb-a2a3-f58fd42c7d57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:26.130360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e1c05-3fab-40cb-a2a3-f58fd42c7d57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:26.131898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e1c05-3fab-40cb-a2a3-f58fd42c7d57 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:26.180612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 857/1717] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:26.180612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e1c05-3fab-40cb-a2a3-f58fd42c7d57 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:26.180768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a151b3ee-8ab5-4a7d-b4e7-0f40afda75f5 None None] MFA Rules not processed for user `f5543fbd902d441cb6f03d3155f1af8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:26.180768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a151b3ee-8ab5-4a7d-b4e7-0f40afda75f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:26.194225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:26.222267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e1c05-3fab-40cb-a2a3-f58fd42c7d57 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:26.253688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2159f37-b212-489a-8da5-1a19c1477025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:26.254377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2159f37-b212-489a-8da5-1a19c1477025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:26.255223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2159f37-b212-489a-8da5-1a19c1477025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:26.255503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2159f37-b212-489a-8da5-1a19c1477025 admin admin] PATH_INFO: `/v3/projects/cdbd86ce99b34d63b53ec02ebcbeeeaf/users/dd1d45a23e4943a5b537d90647983563/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:26.297679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2159f37-b212-489a-8da5-1a19c1477025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdbd86ce99b34d63b53ec02ebcbeeeaf, user_id=dd1d45a23e4943a5b537d90647983563, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:26.302516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2159f37-b212-489a-8da5-1a19c1477025 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:26.385769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 861/1718] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:26.387191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aba60-4896-4ef7-80f6-1bddae77f37e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:26.387476 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aba60-4896-4ef7-80f6-1bddae77f37e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:26.387777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aba60-4896-4ef7-80f6-1bddae77f37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:26.412811 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 858/1719] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:26 2022] PUT /identity/v3/projects/cdbd86ce99b34d63b53ec02ebcbeeeaf/users/dd1d45a23e4943a5b537d90647983563/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:26.414511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922cacc-6ea2-4b08-8934-a363f4b6a44b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:26.414927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922cacc-6ea2-4b08-8934-a363f4b6a44b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:26.415327 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922cacc-6ea2-4b08-8934-a363f4b6a44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:26.462957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-3922cacc-6ea2-4b08-8934-a363f4b6a44b None None] MFA Rules not processed for user `69f976789f894296a86b4cf651fdc3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:26.465190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d16aba60-4896-4ef7-80f6-1bddae77f37e None None] MFA Rules not processed for user `9a61a18900c44bea89647fc9ccdaf110`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:26.465309 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3922cacc-6ea2-4b08-8934-a363f4b6a44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:26.465416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d16aba60-4896-4ef7-80f6-1bddae77f37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:26.633252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 862/1720] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:26.634564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453134-9210-4ba0-9356-83a0dcf2cfe3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:26.634852 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453134-9210-4ba0-9356-83a0dcf2cfe3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:26.635118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453134-9210-4ba0-9356-83a0dcf2cfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:26.684828 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 859/1721] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:26.685379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7e453134-9210-4ba0-9356-83a0dcf2cfe3 None None] MFA Rules not processed for user `f5543fbd902d441cb6f03d3155f1af8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:26.686017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922cacc-6ea2-4b08-8934-a363f4b6a44b None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:26.686250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7e453134-9210-4ba0-9356-83a0dcf2cfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:26.858767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 863/1722] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:26.860240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453134-9210-4ba0-9356-83a0dcf2cfe3 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:26.862053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7e453134-9210-4ba0-9356-83a0dcf2cfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:26.885044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:26.906099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922cacc-6ea2-4b08-8934-a363f4b6a44b None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:26.927973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190d5a8-ad35-4351-abc0-6ba0f6049cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:26.928601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190d5a8-ad35-4351-abc0-6ba0f6049cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:26.928881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190d5a8-ad35-4351-abc0-6ba0f6049cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:26.929202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190d5a8-ad35-4351-abc0-6ba0f6049cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:26.929903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190d5a8-ad35-4351-abc0-6ba0f6049cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:26.931928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190d5a8-ad35-4351-abc0-6ba0f6049cfe admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:26.961692 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 860/1723] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:26.962845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190d5a8-ad35-4351-abc0-6ba0f6049cfe admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:26.980159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:27.055607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453134-9210-4ba0-9356-83a0dcf2cfe3 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:27.081170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e911e1-c307-471d-a745-f9d06233930f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:27.082185 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e911e1-c307-471d-a745-f9d06233930f service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:27.082849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e911e1-c307-471d-a745-f9d06233930f service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:27.083134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e911e1-c307-471d-a745-f9d06233930f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:27.105606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e911e1-c307-471d-a745-f9d06233930f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:27.109156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e911e1-c307-471d-a745-f9d06233930f service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:27.135218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190d5a8-ad35-4351-abc0-6ba0f6049cfe admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:27.161207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38af68-95ca-4aca-bb50-03b9c926ec2f tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf182d5cf73844cfb3fe48d72480f11f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c604eae10f7d41babf230918ef27c585', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:27.161929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38af68-95ca-4aca-bb50-03b9c926ec2f tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:27.162283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38af68-95ca-4aca-bb50-03b9c926ec2f tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:27.162879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38af68-95ca-4aca-bb50-03b9c926ec2f tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:27.166062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38af68-95ca-4aca-bb50-03b9c926ec2f tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:27.166438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38af68-95ca-4aca-bb50-03b9c926ec2f tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:27.167911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 864/1724] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:27.169121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e911e1-c307-471d-a745-f9d06233930f service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:27.169978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38af68-95ca-4aca-bb50-03b9c926ec2f tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:27.172248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f1e911e1-c307-471d-a745-f9d06233930f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:27.209269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 861/1725] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:27.209269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38af68-95ca-4aca-bb50-03b9c926ec2f tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:27.221959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e911e1-c307-471d-a745-f9d06233930f service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:27.229371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38af68-95ca-4aca-bb50-03b9c926ec2f tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:27.242073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59c0fa-eedf-4516-9a14-c4847c89d692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:27.243125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59c0fa-eedf-4516-9a14-c4847c89d692 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:27.243418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59c0fa-eedf-4516-9a14-c4847c89d692 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:27.247314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59c0fa-eedf-4516-9a14-c4847c89d692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:27.257670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af842686-a171-40f3-9933-f85ba6b69c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:27.258428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af842686-a171-40f3-9933-f85ba6b69c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:27.258706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af842686-a171-40f3-9933-f85ba6b69c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:27.258989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af842686-a171-40f3-9933-f85ba6b69c4f admin admin] PATH_INFO: `/v3/projects/cdbd86ce99b34d63b53ec02ebcbeeeaf/users/dd1d45a23e4943a5b537d90647983563/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:27.263651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af842686-a171-40f3-9933-f85ba6b69c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdbd86ce99b34d63b53ec02ebcbeeeaf, user_id=dd1d45a23e4943a5b537d90647983563, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:27.266370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af842686-a171-40f3-9933-f85ba6b69c4f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:27.271804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59c0fa-eedf-4516-9a14-c4847c89d692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:27.273564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59c0fa-eedf-4516-9a14-c4847c89d692 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:27.295734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 865/1726] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:27.314409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59c0fa-eedf-4516-9a14-c4847c89d692 service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:27.318694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 862/1727] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:27 2022] PUT /identity/v3/projects/cdbd86ce99b34d63b53ec02ebcbeeeaf/users/dd1d45a23e4943a5b537d90647983563/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:27.328687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-fb59c0fa-eedf-4516-9a14-c4847c89d692 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:27.330942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af842686-a171-40f3-9933-f85ba6b69c4f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:27.345354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59c0fa-eedf-4516-9a14-c4847c89d692 service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:27.559061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:27.559771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:27.560046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:27.560315 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] PATH_INFO: `/v3/projects/bc9d09b0d78847e5af5665e83dc34be9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:27.563643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc9d09b0d78847e5af5665e83dc34be9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:27.565621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:27.579587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af842686-a171-40f3-9933-f85ba6b69c4f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:27.610283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb5414-8c50-4701-afa8-b37936ed9e31 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf182d5cf73844cfb3fe48d72480f11f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c604eae10f7d41babf230918ef27c585', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:27.611158 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb5414-8c50-4701-afa8-b37936ed9e31 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:27.614615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb5414-8c50-4701-afa8-b37936ed9e31 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:27.614988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb5414-8c50-4701-afa8-b37936ed9e31 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:27.615982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb5414-8c50-4701-afa8-b37936ed9e31 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:27.620368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb5414-8c50-4701-afa8-b37936ed9e31 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:27.723296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 863/1728] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:27 2022] POST /identity/v3/projects => generated 354 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:27.725200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72d20-7097-4e62-8d3c-a5d6d0e6b308 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:27.725589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72d20-7097-4e62-8d3c-a5d6d0e6b308 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:27.726008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72d20-7097-4e62-8d3c-a5d6d0e6b308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:27.772610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc9d09b0d78847e5af5665e83dc34be9', 'request_id': 'req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:27.773447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6fb72d20-7097-4e62-8d3c-a5d6d0e6b308 None None] MFA Rules not processed for user `dd1d45a23e4943a5b537d90647983563`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:27.775441 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6fb72d20-7097-4e62-8d3c-a5d6d0e6b308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:27.804471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc9d09b0d78847e5af5665e83dc34be9', 'request_id': 'req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:27.839631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:27.882963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 864/1729] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:27.884290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb72d20-7097-4e62-8d3c-a5d6d0e6b308 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:27.905094 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc9d09b0d78847e5af5665e83dc34be9', 'request_id': 'req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:27.938640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] The token cache is being invalidate because project bc9d09b0d78847e5af5665e83dc34be9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:27.939802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:27.951574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:27.953537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 866/1730] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:27 2022] DELETE /identity/v3/projects/bc9d09b0d78847e5af5665e83dc34be9 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:27.954346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:27.958201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.029923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb72d20-7097-4e62-8d3c-a5d6d0e6b308 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:28.050610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8747a-f87d-48f3-be64-3d2f40a6da9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:28.051270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8747a-f87d-48f3-be64-3d2f40a6da9d service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.052139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8747a-f87d-48f3-be64-3d2f40a6da9d service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.052464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8747a-f87d-48f3-be64-3d2f40a6da9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.058070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f1a8747a-f87d-48f3-be64-3d2f40a6da9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.079419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:28.123918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d12b4-dc0a-42f3-97e2-f3aae174fdcd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:28.130217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1dcbc-879f-4c39-bc44-28437617818c tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf182d5cf73844cfb3fe48d72480f11f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c604eae10f7d41babf230918ef27c585', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:28.131618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1dcbc-879f-4c39-bc44-28437617818c tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.132594 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1dcbc-879f-4c39-bc44-28437617818c tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.133073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1dcbc-879f-4c39-bc44-28437617818c tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.134647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1dcbc-879f-4c39-bc44-28437617818c tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:28.135175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1dcbc-879f-4c39-bc44-28437617818c tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:28.137592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1dcbc-879f-4c39-bc44-28437617818c tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:28.160235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 867/1731] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:28.160235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1dcbc-879f-4c39-bc44-28437617818c tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:28.162791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9fb1dcbc-879f-4c39-bc44-28437617818c tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.169304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8747a-f87d-48f3-be64-3d2f40a6da9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:28.173825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8747a-f87d-48f3-be64-3d2f40a6da9d service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:28.212861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 865/1732] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:28.214269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f507c-0458-4ece-bcc8-b8350f27b0e9 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.215324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f507c-0458-4ece-bcc8-b8350f27b0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.216116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f507c-0458-4ece-bcc8-b8350f27b0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.254218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d01f507c-0458-4ece-bcc8-b8350f27b0e9 None None] MFA Rules not processed for user `de2273220e824339bd13d968c75df26b`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:28.255895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d01f507c-0458-4ece-bcc8-b8350f27b0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.307416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1dcbc-879f-4c39-bc44-28437617818c tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:28.318774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafc212-ac53-40d7-97ba-62f29eb23d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:28.319407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafc212-ac53-40d7-97ba-62f29eb23d16 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.319712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafc212-ac53-40d7-97ba-62f29eb23d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.320568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafc212-ac53-40d7-97ba-62f29eb23d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.322685 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-caafc212-ac53-40d7-97ba-62f29eb23d16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.351897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafc212-ac53-40d7-97ba-62f29eb23d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:28.354098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafc212-ac53-40d7-97ba-62f29eb23d16 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:28.355999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 866/1733] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:28.358674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67531f01-814e-427c-93a2-87d64c6e5b1d None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.359959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67531f01-814e-427c-93a2-87d64c6e5b1d None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.360317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67531f01-814e-427c-93a2-87d64c6e5b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.371271 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 868/1734] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:28.375082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2738055-312d-4f58-a1f0-262d05d164fd None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.375082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2738055-312d-4f58-a1f0-262d05d164fd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.375293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2738055-312d-4f58-a1f0-262d05d164fd None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.378299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 869/1735] 10.222.0.62 () {54 vars in 912 bytes} [Mon Sep 5 12:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:29:28.379063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2738055-312d-4f58-a1f0-262d05d164fd None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:28.392363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-67531f01-814e-427c-93a2-87d64c6e5b1d None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:28.393875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-67531f01-814e-427c-93a2-87d64c6e5b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.394388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2738055-312d-4f58-a1f0-262d05d164fd None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:28.400785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf182d5cf73844cfb3fe48d72480f11f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c604eae10f7d41babf230918ef27c585', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:28.401997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.402283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.402556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] PATH_INFO: `/v3/projects/ca63d5065e1340f4a3df10be96442201` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.403805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ca63d5065e1340f4a3df10be96442201)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:28.406830 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:28.409691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:28.466781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 867/1736] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:28.468174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a03b4d-9821-4d94-bcb9-6089e0ee612a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.468465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a03b4d-9821-4d94-bcb9-6089e0ee612a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.468773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a03b4d-9821-4d94-bcb9-6089e0ee612a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.540141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-31a03b4d-9821-4d94-bcb9-6089e0ee612a None None] MFA Rules not processed for user `9a89a2e6aeb4495a84a61bd82b02bfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:28.540141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-31a03b4d-9821-4d94-bcb9-6089e0ee612a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.610430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca63d5065e1340f4a3df10be96442201', 'request_id': 'req-ca186cfe-b831-42c9-9e7b-5906ea434072'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:28.627398 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca63d5065e1340f4a3df10be96442201', 'request_id': 'req-ca186cfe-b831-42c9-9e7b-5906ea434072'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:28.663189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 868/1737] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:28.663189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e61b6-c284-4acb-a304-d824623f4858 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.663189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e61b6-c284-4acb-a304-d824623f4858 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.663189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e61b6-c284-4acb-a304-d824623f4858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.703583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-145e61b6-c284-4acb-a304-d824623f4858 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:28.706989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-145e61b6-c284-4acb-a304-d824623f4858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.710590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca63d5065e1340f4a3df10be96442201', 'request_id': 'req-ca186cfe-b831-42c9-9e7b-5906ea434072'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:28.730083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] The token cache is being invalidate because project ca63d5065e1340f4a3df10be96442201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:28.730376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:28.734049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:28.736743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 870/1738] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:28 2022] DELETE /identity/v3/projects/ca63d5065e1340f4a3df10be96442201 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:28.743835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:28.748609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.844525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca186cfe-b831-42c9-9e7b-5906ea434072 tempest-VolumeQuotaClassesTest-1149819354 tempest-VolumeQuotaClassesTest-1149819354-project-admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:28.854394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b978f-9b11-4bee-8cd1-31413a99dc15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:28.855034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b978f-9b11-4bee-8cd1-31413a99dc15 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.855357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b978f-9b11-4bee-8cd1-31413a99dc15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.855716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b978f-9b11-4bee-8cd1-31413a99dc15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.868120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-299b978f-9b11-4bee-8cd1-31413a99dc15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.884498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b978f-9b11-4bee-8cd1-31413a99dc15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:28.885940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b978f-9b11-4bee-8cd1-31413a99dc15 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:28.899314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 869/1739] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:28.900380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e61b6-c284-4acb-a304-d824623f4858 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:28.904082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-145e61b6-c284-4acb-a304-d824623f4858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:28.926041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 871/1740] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:28.927270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b978f-9b11-4bee-8cd1-31413a99dc15 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:28.954965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b978f-9b11-4bee-8cd1-31413a99dc15 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:28.968092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecc11b-6c3b-4ea0-92e4-d208b386cdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:28.969055 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecc11b-6c3b-4ea0-92e4-d208b386cdf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:28.969335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecc11b-6c3b-4ea0-92e4-d208b386cdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:28.969605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecc11b-6c3b-4ea0-92e4-d208b386cdf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:28.970253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecc11b-6c3b-4ea0-92e4-d208b386cdf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:28.970848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecc11b-6c3b-4ea0-92e4-d208b386cdf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:28.974830 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecc11b-6c3b-4ea0-92e4-d208b386cdf2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:28.988428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:29.005758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 872/1741] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:29.007439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fa2e2-2e0b-415c-90cb-ea9d673d6908 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:29.011697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fa2e2-2e0b-415c-90cb-ea9d673d6908 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:29.011697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fa2e2-2e0b-415c-90cb-ea9d673d6908 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:29.015822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 873/1742] 10.222.0.62 () {54 vars in 912 bytes} [Mon Sep 5 12:29:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:29:29.016733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fa2e2-2e0b-415c-90cb-ea9d673d6908 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:29.038368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fa2e2-2e0b-415c-90cb-ea9d673d6908 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:29.051889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e61b6-c284-4acb-a304-d824623f4858 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:29.053167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d65a1-6c42-45d8-84eb-799b40b023af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:29.053472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d65a1-6c42-45d8-84eb-799b40b023af admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:29.053755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d65a1-6c42-45d8-84eb-799b40b023af admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:29.054028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d65a1-6c42-45d8-84eb-799b40b023af admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:29.054910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d65a1-6c42-45d8-84eb-799b40b023af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:29.058278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d65a1-6c42-45d8-84eb-799b40b023af admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:29.075852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1836e-a5ce-4296-8821-b68a620c3d7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:29.077483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1836e-a5ce-4296-8821-b68a620c3d7e service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:29.079879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1836e-a5ce-4296-8821-b68a620c3d7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:29.084075 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1836e-a5ce-4296-8821-b68a620c3d7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:29.091379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5de1836e-a5ce-4296-8821-b68a620c3d7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:29.127493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:29.227376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 874/1743] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:29 2022] POST /identity/v3/projects => generated 365 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:29.228719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95377c3c-c2bc-4f65-b78e-609c1078b6ed None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:29.228987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95377c3c-c2bc-4f65-b78e-609c1078b6ed None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:29.229255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95377c3c-c2bc-4f65-b78e-609c1078b6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:29.274575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1836e-a5ce-4296-8821-b68a620c3d7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:29.279871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1836e-a5ce-4296-8821-b68a620c3d7e service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:29.315057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 870/1744] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:29.316267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1836e-a5ce-4296-8821-b68a620c3d7e service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:29.318148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-95377c3c-c2bc-4f65-b78e-609c1078b6ed None None] MFA Rules not processed for user `9a89a2e6aeb4495a84a61bd82b02bfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:29.323347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-95377c3c-c2bc-4f65-b78e-609c1078b6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:29.356007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1836e-a5ce-4296-8821-b68a620c3d7e service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:29.446140 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 875/1745] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:29:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:29.460418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95377c3c-c2bc-4f65-b78e-609c1078b6ed None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:29.462077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-95377c3c-c2bc-4f65-b78e-609c1078b6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:29.474005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95377c3c-c2bc-4f65-b78e-609c1078b6ed None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:29.475606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c16761-06ac-42c3-9c8d-40df468ae73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:29.475813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c16761-06ac-42c3-9c8d-40df468ae73b admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:29.476183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c16761-06ac-42c3-9c8d-40df468ae73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:29.476526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c16761-06ac-42c3-9c8d-40df468ae73b admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:29.477705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c16761-06ac-42c3-9c8d-40df468ae73b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:29.479305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c16761-06ac-42c3-9c8d-40df468ae73b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:29.489405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489e469-e28f-456b-8a16-94ff22ab9e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:29.490277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489e469-e28f-456b-8a16-94ff22ab9e4c service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:29.490557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489e469-e28f-456b-8a16-94ff22ab9e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:29.490825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489e469-e28f-456b-8a16-94ff22ab9e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:29.500736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489e469-e28f-456b-8a16-94ff22ab9e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:29.502759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489e469-e28f-456b-8a16-94ff22ab9e4c service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:29.512886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 876/1746] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:29:29.638588 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 871/1747] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:29 2022] POST /identity/v3/users => generated 283 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:29.644568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489e469-e28f-456b-8a16-94ff22ab9e4c service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:29.659630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489e469-e28f-456b-8a16-94ff22ab9e4c service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:29.680786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeef534-578f-45e1-b97f-4df144a6e3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:29.681011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeef534-578f-45e1-b97f-4df144a6e3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:29.681335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeef534-578f-45e1-b97f-4df144a6e3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:29.681610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeef534-578f-45e1-b97f-4df144a6e3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:29.682253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeef534-578f-45e1-b97f-4df144a6e3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:29.695380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeef534-578f-45e1-b97f-4df144a6e3f6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:29.731353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 877/1748] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:29.735181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c16761-06ac-42c3-9c8d-40df468ae73b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:29.754156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c16761-06ac-42c3-9c8d-40df468ae73b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:29.774903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe47a6-d1b7-4629-834d-3381c5d79d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:29.775578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe47a6-d1b7-4629-834d-3381c5d79d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:29.775882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe47a6-d1b7-4629-834d-3381c5d79d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:29.777595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe47a6-d1b7-4629-834d-3381c5d79d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:29.777595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe47a6-d1b7-4629-834d-3381c5d79d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:29.778739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe47a6-d1b7-4629-834d-3381c5d79d1b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:29.814173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 872/1749] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:29.815386 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeef534-578f-45e1-b97f-4df144a6e3f6 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:29.848091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeef534-578f-45e1-b97f-4df144a6e3f6 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:29.848091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2d94a-5be6-496d-b091-fe911080be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:29.849496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2d94a-5be6-496d-b091-fe911080be7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:29.850079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2d94a-5be6-496d-b091-fe911080be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:29.850079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2d94a-5be6-496d-b091-fe911080be7f admin admin] PATH_INFO: `/v3/projects/0edb0312c74d49d0a6455808365a5315/users/e46ee1ee447e44b4b8efa3b2fed4a5b1/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:29.879001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2d94a-5be6-496d-b091-fe911080be7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0edb0312c74d49d0a6455808365a5315, user_id=e46ee1ee447e44b4b8efa3b2fed4a5b1, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:29.891603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2d94a-5be6-496d-b091-fe911080be7f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:29.921573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe47a6-d1b7-4629-834d-3381c5d79d1b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:29.923876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cbbe47a6-d1b7-4629-834d-3381c5d79d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:30.000499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 878/1750] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:29 2022] PUT /identity/v3/projects/0edb0312c74d49d0a6455808365a5315/users/e46ee1ee447e44b4b8efa3b2fed4a5b1/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:30.014527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa9bdc-3f6e-4a78-a388-8fc1c36cf85c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:30.022310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa9bdc-3f6e-4a78-a388-8fc1c36cf85c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:30.023143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa9bdc-3f6e-4a78-a388-8fc1c36cf85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:30.142811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d9fa9bdc-3f6e-4a78-a388-8fc1c36cf85c None None] MFA Rules not processed for user `e46ee1ee447e44b4b8efa3b2fed4a5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:30.146100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d9fa9bdc-3f6e-4a78-a388-8fc1c36cf85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:30.200728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe47a6-d1b7-4629-834d-3381c5d79d1b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:30.307029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 879/1751] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:30.318185 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa9bdc-3f6e-4a78-a388-8fc1c36cf85c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:30.324193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d9fa9bdc-3f6e-4a78-a388-8fc1c36cf85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:30.342589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:30.391769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:30.428861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e192fed-f4d4-42b2-90ff-22bcdb094b98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:30.429602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e192fed-f4d4-42b2-90ff-22bcdb094b98 service placement] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:30.429800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e192fed-f4d4-42b2-90ff-22bcdb094b98 service placement] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:30.430059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e192fed-f4d4-42b2-90ff-22bcdb094b98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:30.432085 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2e192fed-f4d4-42b2-90ff-22bcdb094b98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:30.475138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e192fed-f4d4-42b2-90ff-22bcdb094b98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:30.476728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e192fed-f4d4-42b2-90ff-22bcdb094b98 service placement] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:30.511672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 873/1752] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:29:30.513005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e192fed-f4d4-42b2-90ff-22bcdb094b98 service placement] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:30.532108 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa9bdc-3f6e-4a78-a388-8fc1c36cf85c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:30.535885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e192fed-f4d4-42b2-90ff-22bcdb094b98 service placement] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:30.542624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:30.543285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:30.545778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:30.545778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] PATH_INFO: `/v3/users/fa1d3fd47f7b4f80a707e847305dd080` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:30.550273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa1d3fd47f7b4f80a707e847305dd080)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:30.555882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:30.556276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342d9ff-0972-430a-8986-a9b0708de0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:30.556921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342d9ff-0972-430a-8986-a9b0708de0ad admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:30.557203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342d9ff-0972-430a-8986-a9b0708de0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:30.557479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342d9ff-0972-430a-8986-a9b0708de0ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:30.558185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342d9ff-0972-430a-8986-a9b0708de0ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:30.567896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342d9ff-0972-430a-8986-a9b0708de0ad admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:30.740276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 874/1753] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:30 2022] POST /identity/v3/projects => generated 363 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:30.748562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342d9ff-0972-430a-8986-a9b0708de0ad admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:30.861519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa1d3fd47f7b4f80a707e847305dd080'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:30.865910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa1d3fd47f7b4f80a707e847305dd080'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:30.911208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342d9ff-0972-430a-8986-a9b0708de0ad admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:30.929610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1f7d1-5372-4928-8081-c97dde468188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:30.929610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1f7d1-5372-4928-8081-c97dde468188 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:30.929610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1f7d1-5372-4928-8081-c97dde468188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:30.929610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1f7d1-5372-4928-8081-c97dde468188 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:30.931288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1f7d1-5372-4928-8081-c97dde468188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:30.934958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1f7d1-5372-4928-8081-c97dde468188 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:30.948842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa1d3fd47f7b4f80a707e847305dd080'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:30.977675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa1d3fd47f7b4f80a707e847305dd080'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:30.977675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fe2cac3e-5b37-4c32-bf43-2785f80d4aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa1d3fd47f7b4f80a707e847305dd080'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:31.001073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 880/1754] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:30 2022] DELETE /identity/v3/users/fa1d3fd47f7b4f80a707e847305dd080 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:31.006369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eaf85b-1f32-4e9a-9599-83c5a05a8f1f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:31.012365 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eaf85b-1f32-4e9a-9599-83c5a05a8f1f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:31.012735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eaf85b-1f32-4e9a-9599-83c5a05a8f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:31.055906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 875/1755] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:30 2022] POST /identity/v3/users => generated 288 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:31.060067 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1f7d1-5372-4928-8081-c97dde468188 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:31.064478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-22b1f7d1-5372-4928-8081-c97dde468188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:31.084294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d0eaf85b-1f32-4e9a-9599-83c5a05a8f1f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:31.086003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d0eaf85b-1f32-4e9a-9599-83c5a05a8f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:31.096446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1f7d1-5372-4928-8081-c97dde468188 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:31.360291 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f340-bd36-41e6-a35e-f253c8d3d9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:31.360903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f340-bd36-41e6-a35e-f253c8d3d9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:31.361180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f340-bd36-41e6-a35e-f253c8d3d9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:31.361459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f340-bd36-41e6-a35e-f253c8d3d9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:31.362124 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883f340-bd36-41e6-a35e-f253c8d3d9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:31.363816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883f340-bd36-41e6-a35e-f253c8d3d9a9 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:31.370129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 881/1756] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:31.389736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eaf85b-1f32-4e9a-9599-83c5a05a8f1f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:31.391880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d0eaf85b-1f32-4e9a-9599-83c5a05a8f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:31.410237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 876/1757] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:29:31.412259 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f340-bd36-41e6-a35e-f253c8d3d9a9 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:31.447912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f340-bd36-41e6-a35e-f253c8d3d9a9 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:31.469893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f5bd0-f373-4877-ad04-d8412df06d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:31.470585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f5bd0-f373-4877-ad04-d8412df06d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:31.470866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f5bd0-f373-4877-ad04-d8412df06d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:31.471222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f5bd0-f373-4877-ad04-d8412df06d99 admin admin] PATH_INFO: `/v3/projects/6e3209e42263420c81a04c878809ad9f/users/6c561664d1a74586b438804856931412/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:31.524292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f5bd0-f373-4877-ad04-d8412df06d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3209e42263420c81a04c878809ad9f, user_id=6c561664d1a74586b438804856931412, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:31.527832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f5bd0-f373-4877-ad04-d8412df06d99 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:31.578221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eaf85b-1f32-4e9a-9599-83c5a05a8f1f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:31.587651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159af44-51a6-4cd4-a3bb-55e408ae4320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:31.588434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159af44-51a6-4cd4-a3bb-55e408ae4320 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:31.588723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159af44-51a6-4cd4-a3bb-55e408ae4320 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:31.589154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159af44-51a6-4cd4-a3bb-55e408ae4320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:31.622559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159af44-51a6-4cd4-a3bb-55e408ae4320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:31.625631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159af44-51a6-4cd4-a3bb-55e408ae4320 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:31.632184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 877/1758] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:31 2022] PUT /identity/v3/projects/6e3209e42263420c81a04c878809ad9f/users/6c561664d1a74586b438804856931412/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:29:31.640050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 882/1759] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:29:31.645040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f5bd0-f373-4877-ad04-d8412df06d99 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:31.648898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f1c24-5403-48a5-b0af-9478316ecd69 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:31.649181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f1c24-5403-48a5-b0af-9478316ecd69 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:31.649453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f1c24-5403-48a5-b0af-9478316ecd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:31.692630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a33f1c24-5403-48a5-b0af-9478316ecd69 None None] MFA Rules not processed for user `f5543fbd902d441cb6f03d3155f1af8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:31.695851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a33f1c24-5403-48a5-b0af-9478316ecd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:31.806721 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 883/1760] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:31.844432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f5bd0-f373-4877-ad04-d8412df06d99 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:31.862445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a4b4f-8811-4262-aedc-c7141dc2da1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:31.863195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a4b4f-8811-4262-aedc-c7141dc2da1d admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:31.863719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a4b4f-8811-4262-aedc-c7141dc2da1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:31.864214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a4b4f-8811-4262-aedc-c7141dc2da1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:31.865210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a4b4f-8811-4262-aedc-c7141dc2da1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:31.869489 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a4b4f-8811-4262-aedc-c7141dc2da1d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:31.906681 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f1c24-5403-48a5-b0af-9478316ecd69 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:31.912205 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 878/1761] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:29:31.948487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f1c24-5403-48a5-b0af-9478316ecd69 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:31.954304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a4b4f-8811-4262-aedc-c7141dc2da1d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:31.957167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e03a4b4f-8811-4262-aedc-c7141dc2da1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:31.970814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5559292-e047-4e4e-a35b-6fe615af2379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:31.970814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5559292-e047-4e4e-a35b-6fe615af2379 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:31.970814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5559292-e047-4e4e-a35b-6fe615af2379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:31.970814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5559292-e047-4e4e-a35b-6fe615af2379 admin admin] PATH_INFO: `/v3/projects/6e3209e42263420c81a04c878809ad9f/users/6c561664d1a74586b438804856931412/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:31.971211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5559292-e047-4e4e-a35b-6fe615af2379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3209e42263420c81a04c878809ad9f, user_id=6c561664d1a74586b438804856931412, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:31.974156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5559292-e047-4e4e-a35b-6fe615af2379 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:32.001072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 884/1762] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:31 2022] PUT /identity/v3/projects/6e3209e42263420c81a04c878809ad9f/users/6c561664d1a74586b438804856931412/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:32.014291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee2e11-8342-40d2-8f2d-cd1a0cb8d11c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:32.014773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee2e11-8342-40d2-8f2d-cd1a0cb8d11c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:32.015318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee2e11-8342-40d2-8f2d-cd1a0cb8d11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:32.057160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ffee2e11-8342-40d2-8f2d-cd1a0cb8d11c None None] MFA Rules not processed for user `6c561664d1a74586b438804856931412`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:32.060116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ffee2e11-8342-40d2-8f2d-cd1a0cb8d11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:32.086721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:32.122597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:32.124403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a4b4f-8811-4262-aedc-c7141dc2da1d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:32.230632 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 885/1763] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:29:32.249357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee2e11-8342-40d2-8f2d-cd1a0cb8d11c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:32.251103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ffee2e11-8342-40d2-8f2d-cd1a0cb8d11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:32.268880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63506c30-a630-4794-8114-0118e76373d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:32.269540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63506c30-a630-4794-8114-0118e76373d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:32.269817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63506c30-a630-4794-8114-0118e76373d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:32.270096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63506c30-a630-4794-8114-0118e76373d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:32.278059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-63506c30-a630-4794-8114-0118e76373d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:32.371251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee2e11-8342-40d2-8f2d-cd1a0cb8d11c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:32.375833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2e92f-563d-4bd2-bb1e-3e74d3ca7b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:32.376459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2e92f-563d-4bd2-bb1e-3e74d3ca7b6e service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:32.376781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2e92f-563d-4bd2-bb1e-3e74d3ca7b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:32.377148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2e92f-563d-4bd2-bb1e-3e74d3ca7b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:32.404513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2e92f-563d-4bd2-bb1e-3e74d3ca7b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:32.405954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2e92f-563d-4bd2-bb1e-3e74d3ca7b6e service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:32.408136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63506c30-a630-4794-8114-0118e76373d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:32.409738 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63506c30-a630-4794-8114-0118e76373d8 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:32.420766 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 879/1764] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:32.427404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 886/1765] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:33.270696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63506c30-a630-4794-8114-0118e76373d8 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:33.272204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-63506c30-a630-4794-8114-0118e76373d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:33.290633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:33.427890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63506c30-a630-4794-8114-0118e76373d8 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:33.453872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:33.454791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:33.454981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:33.455404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] PATH_INFO: `/v3/projects/da505fda26fa4cb0ac5f011abb170010` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:33.458506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da505fda26fa4cb0ac5f011abb170010)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:33.462444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:33.603119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bcb20-884c-4c41-b088-2fa90627d387 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:33.603691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bcb20-884c-4c41-b088-2fa90627d387 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:33.603770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bcb20-884c-4c41-b088-2fa90627d387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:33.646035 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1d2bcb20-884c-4c41-b088-2fa90627d387 None None] MFA Rules not processed for user `9a61a18900c44bea89647fc9ccdaf110`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:33.647438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1d2bcb20-884c-4c41-b088-2fa90627d387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:33.681108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da505fda26fa4cb0ac5f011abb170010', 'request_id': 'req-831e2919-3f0e-4939-a4b2-8158882128a7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:33.694389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da505fda26fa4cb0ac5f011abb170010', 'request_id': 'req-831e2919-3f0e-4939-a4b2-8158882128a7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:33.791325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 887/1766] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:33.848403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da505fda26fa4cb0ac5f011abb170010', 'request_id': 'req-831e2919-3f0e-4939-a4b2-8158882128a7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:33.878302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] The token cache is being invalidate because project da505fda26fa4cb0ac5f011abb170010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:33.880781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:33.881619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:33.888485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 880/1767] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:33 2022] DELETE /identity/v3/projects/da505fda26fa4cb0ac5f011abb170010 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:33.891441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bcb20-884c-4c41-b088-2fa90627d387 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:33.918439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:34.029235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bcb20-884c-4c41-b088-2fa90627d387 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:34.056699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:34.057957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:34.058244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:34.061113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] PATH_INFO: `/v3/users/bf182d5cf73844cfb3fe48d72480f11f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:34.065657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf182d5cf73844cfb3fe48d72480f11f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:34.073484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:34.421900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf182d5cf73844cfb3fe48d72480f11f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:34.422892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf182d5cf73844cfb3fe48d72480f11f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:34.562003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf182d5cf73844cfb3fe48d72480f11f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:34.576858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf182d5cf73844cfb3fe48d72480f11f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:34.577573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-53b720de-39db-4b83-ab2d-2700a5251772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf182d5cf73844cfb3fe48d72480f11f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:34.595504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:34.599076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:34.599360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 888/1768] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:33 2022] DELETE /identity/v3/users/bf182d5cf73844cfb3fe48d72480f11f => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:34.626435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e2919-3f0e-4939-a4b2-8158882128a7 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:34.697855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:34.699054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:34.699482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:34.699994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] PATH_INFO: `/v3/users/e46ee1ee447e44b4b8efa3b2fed4a5b1` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:34.701615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46ee1ee447e44b4b8efa3b2fed4a5b1)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:34.704884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:35.056581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:35.079181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46ee1ee447e44b4b8efa3b2fed4a5b1'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:35.079181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46ee1ee447e44b4b8efa3b2fed4a5b1'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:35.186190 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46ee1ee447e44b4b8efa3b2fed4a5b1'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:35.208706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46ee1ee447e44b4b8efa3b2fed4a5b1'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:35.210348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46ee1ee447e44b4b8efa3b2fed4a5b1'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:35.227747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 881/1769] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:34 2022] DELETE /identity/v3/users/e46ee1ee447e44b4b8efa3b2fed4a5b1 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:35.237361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fa40c-0996-48dc-a4e5-23dfb50ca615 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:35.237904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fa40c-0996-48dc-a4e5-23dfb50ca615 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:35.237904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fa40c-0996-48dc-a4e5-23dfb50ca615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:35.258972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-570fa40c-0996-48dc-a4e5-23dfb50ca615 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:35.260394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-570fa40c-0996-48dc-a4e5-23dfb50ca615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:35.328048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 889/1770] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:35.335711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:35.338918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:35.382636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fa40c-0996-48dc-a4e5-23dfb50ca615 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:35.389119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-570fa40c-0996-48dc-a4e5-23dfb50ca615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:35.414258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fa40c-0996-48dc-a4e5-23dfb50ca615 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:35.425992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:35.427150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:35.427428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:35.427728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] PATH_INFO: `/v3/projects/c604eae10f7d41babf230918ef27c585` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:35.432559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c604eae10f7d41babf230918ef27c585)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:35.434530 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:35.440451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdb54c-298e-4821-bb8e-9e89a0da4c6a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:35.445554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84690f4-fb53-40bd-92e0-82a329dd51ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:35.446173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84690f4-fb53-40bd-92e0-82a329dd51ba service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:35.446452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84690f4-fb53-40bd-92e0-82a329dd51ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:35.446723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84690f4-fb53-40bd-92e0-82a329dd51ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:35.472980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84690f4-fb53-40bd-92e0-82a329dd51ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:35.475094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84690f4-fb53-40bd-92e0-82a329dd51ba service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:35.493231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 882/1771] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:35.574881 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:35.633134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c604eae10f7d41babf230918ef27c585', 'request_id': 'req-c3e43e07-3ad3-4285-b070-a421c88344c5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:35.639565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c604eae10f7d41babf230918ef27c585', 'request_id': 'req-c3e43e07-3ad3-4285-b070-a421c88344c5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:35.785490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c604eae10f7d41babf230918ef27c585', 'request_id': 'req-c3e43e07-3ad3-4285-b070-a421c88344c5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:35.802615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] The token cache is being invalidate because project c604eae10f7d41babf230918ef27c585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:35.802912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:35.803689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c3e43e07-3ad3-4285-b070-a421c88344c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:35.805326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 890/1772] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:35 2022] DELETE /identity/v3/projects/c604eae10f7d41babf230918ef27c585 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:35.811671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce8860-0390-4482-9325-8a6e13dc86cb None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:35.812165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce8860-0390-4482-9325-8a6e13dc86cb None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:35.812273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce8860-0390-4482-9325-8a6e13dc86cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:35.857690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-94ce8860-0390-4482-9325-8a6e13dc86cb None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:35.860447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-94ce8860-0390-4482-9325-8a6e13dc86cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:36.006316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 883/1773] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:36.016116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e82b6-f728-4af4-8508-eb3643646fb5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:36.016379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e82b6-f728-4af4-8508-eb3643646fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:36.017374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e82b6-f728-4af4-8508-eb3643646fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:36.077485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-616e82b6-f728-4af4-8508-eb3643646fb5 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:36.079133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-616e82b6-f728-4af4-8508-eb3643646fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:36.095649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 891/1774] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:36.096202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce8860-0390-4482-9325-8a6e13dc86cb None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:36.110215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce8860-0390-4482-9325-8a6e13dc86cb None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:36.123821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb086c74-60d3-448a-98f6-437140c59f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:36.125102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb086c74-60d3-448a-98f6-437140c59f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:36.125562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb086c74-60d3-448a-98f6-437140c59f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:36.125983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb086c74-60d3-448a-98f6-437140c59f05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:36.126787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb086c74-60d3-448a-98f6-437140c59f05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:36.127935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb086c74-60d3-448a-98f6-437140c59f05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:36.131301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb086c74-60d3-448a-98f6-437140c59f05 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:36.162650 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 884/1775] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:36.168933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e82b6-f728-4af4-8508-eb3643646fb5 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:36.181467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e82b6-f728-4af4-8508-eb3643646fb5 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:36.192996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178a952-ff82-4ab0-aed3-e2c54b6bb23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:36.203640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a952-ff82-4ab0-aed3-e2c54b6bb23c admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:36.203640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a952-ff82-4ab0-aed3-e2c54b6bb23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:36.203640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a952-ff82-4ab0-aed3-e2c54b6bb23c admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:36.203640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178a952-ff82-4ab0-aed3-e2c54b6bb23c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:36.203640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178a952-ff82-4ab0-aed3-e2c54b6bb23c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:36.426766 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 892/1776] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:36 2022] POST /identity/v3/projects => generated 409 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:36.432016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb086c74-60d3-448a-98f6-437140c59f05 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:36.445679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb086c74-60d3-448a-98f6-437140c59f05 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:36.537705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28de40e-58ba-4122-a46e-a39baff9a5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:36.538655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28de40e-58ba-4122-a46e-a39baff9a5ac admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:36.538992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28de40e-58ba-4122-a46e-a39baff9a5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:36.547472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28de40e-58ba-4122-a46e-a39baff9a5ac admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:36.548299 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28de40e-58ba-4122-a46e-a39baff9a5ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:36.550354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28de40e-58ba-4122-a46e-a39baff9a5ac admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:36.666739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 885/1777] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:36 2022] POST /identity/v3/users => generated 305 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:36.668313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178a952-ff82-4ab0-aed3-e2c54b6bb23c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:36.682979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178a952-ff82-4ab0-aed3-e2c54b6bb23c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:36.699071 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28de40e-58ba-4122-a46e-a39baff9a5ac admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:36.702189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f28de40e-58ba-4122-a46e-a39baff9a5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:36.706883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e246b-5a39-40a4-a1ef-94ed6741a563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:36.708464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e246b-5a39-40a4-a1ef-94ed6741a563 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:36.712494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e246b-5a39-40a4-a1ef-94ed6741a563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:36.712962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e246b-5a39-40a4-a1ef-94ed6741a563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:36.719491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e246b-5a39-40a4-a1ef-94ed6741a563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:36.724921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28de40e-58ba-4122-a46e-a39baff9a5ac admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:36.729248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e246b-5a39-40a4-a1ef-94ed6741a563 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:36.749147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:36.750647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:36.753853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:36.753853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] PATH_INFO: `/v3/projects/0edb0312c74d49d0a6455808365a5315` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:36.753853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0edb0312c74d49d0a6455808365a5315)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:36.758711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:36.783281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 893/1778] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:36.787289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e246b-5a39-40a4-a1ef-94ed6741a563 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:36.851448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e246b-5a39-40a4-a1ef-94ed6741a563 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:36.882240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ea93e-05af-4fe1-b89c-648c6cf7c6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:36.882957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ea93e-05af-4fe1-b89c-648c6cf7c6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:36.883441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ea93e-05af-4fe1-b89c-648c6cf7c6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:36.884036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ea93e-05af-4fe1-b89c-648c6cf7c6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:36.884892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ea93e-05af-4fe1-b89c-648c6cf7c6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:36.892524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ea93e-05af-4fe1-b89c-648c6cf7c6b8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:36.915483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 894/1779] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:36.920023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ea93e-05af-4fe1-b89c-648c6cf7c6b8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:36.943868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ea93e-05af-4fe1-b89c-648c6cf7c6b8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:36.958735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c95f6-d0e5-41e7-b038-1cea25e56224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:36.963828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c95f6-d0e5-41e7-b038-1cea25e56224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:36.963828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c95f6-d0e5-41e7-b038-1cea25e56224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:36.963828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c95f6-d0e5-41e7-b038-1cea25e56224 admin admin] PATH_INFO: `/v3/projects/41cdf50f1b0a4256b101c94d735a8998/users/c96ec793badf41b4b6c9c39807f848c7/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:36.980149 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c95f6-d0e5-41e7-b038-1cea25e56224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cdf50f1b0a4256b101c94d735a8998, user_id=c96ec793badf41b4b6c9c39807f848c7, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:36.983443 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c95f6-d0e5-41e7-b038-1cea25e56224 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:37.131399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 895/1780] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:36 2022] PUT /identity/v3/projects/41cdf50f1b0a4256b101c94d735a8998/users/c96ec793badf41b4b6c9c39807f848c7/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:37.150887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d8d5-cf45-40eb-b0dd-d542915bd5c8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:37.158244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d8d5-cf45-40eb-b0dd-d542915bd5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:37.158450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d8d5-cf45-40eb-b0dd-d542915bd5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:37.164559 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0edb0312c74d49d0a6455808365a5315', 'request_id': 'req-b3b87060-4555-474f-9841-93304a4744bc'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.181192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0edb0312c74d49d0a6455808365a5315', 'request_id': 'req-b3b87060-4555-474f-9841-93304a4744bc'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.228421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5499d8d5-cf45-40eb-b0dd-d542915bd5c8 None None] MFA Rules not processed for user `c96ec793badf41b4b6c9c39807f848c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:37.229793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5499d8d5-cf45-40eb-b0dd-d542915bd5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:37.305046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0edb0312c74d49d0a6455808365a5315', 'request_id': 'req-b3b87060-4555-474f-9841-93304a4744bc'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.327103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] The token cache is being invalidate because project 0edb0312c74d49d0a6455808365a5315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:37.327396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.329336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3b87060-4555-474f-9841-93304a4744bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.331854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 886/1781] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:36 2022] DELETE /identity/v3/projects/0edb0312c74d49d0a6455808365a5315 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:37.336258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2e1de-8f71-49af-9b9a-37767097424e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:37.337030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2e1de-8f71-49af-9b9a-37767097424e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:37.338352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2e1de-8f71-49af-9b9a-37767097424e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:37.362492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 896/1782] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:37 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:37.363865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499d8d5-cf45-40eb-b0dd-d542915bd5c8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:37.492610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-22c2e1de-8f71-49af-9b9a-37767097424e None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:37.496866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-22c2e1de-8f71-49af-9b9a-37767097424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:37.534345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499d8d5-cf45-40eb-b0dd-d542915bd5c8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:37.553596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:37.554249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:37.554524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:37.554792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] PATH_INFO: `/v3/users/6c561664d1a74586b438804856931412` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:37.573750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c561664d1a74586b438804856931412)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:37.580592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:37.593276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 887/1783] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:29:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:37.594353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2e1de-8f71-49af-9b9a-37767097424e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:37.616522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-22c2e1de-8f71-49af-9b9a-37767097424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:37.633861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2e1de-8f71-49af-9b9a-37767097424e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:37.647357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61932381-ed02-4040-8486-14e8770847ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:37.650076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61932381-ed02-4040-8486-14e8770847ce admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:37.650076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61932381-ed02-4040-8486-14e8770847ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:37.650076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61932381-ed02-4040-8486-14e8770847ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:37.650076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61932381-ed02-4040-8486-14e8770847ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:37.652039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61932381-ed02-4040-8486-14e8770847ce admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:37.828941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 888/1784] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:37 2022] POST /identity/v3/projects => generated 409 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:37.833194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61932381-ed02-4040-8486-14e8770847ce admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:37.889498 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:37.938273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61932381-ed02-4040-8486-14e8770847ce admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:37.951937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7bda3-e0a2-448e-a60a-d63321ee88fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:37.953311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7bda3-e0a2-448e-a60a-d63321ee88fd admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:37.953690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7bda3-e0a2-448e-a60a-d63321ee88fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:37.954060 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7bda3-e0a2-448e-a60a-d63321ee88fd admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:37.954867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7bda3-e0a2-448e-a60a-d63321ee88fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:37.957352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7bda3-e0a2-448e-a60a-d63321ee88fd admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:37.970252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c561664d1a74586b438804856931412'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:37.971321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c561664d1a74586b438804856931412'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:38.102269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c561664d1a74586b438804856931412'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:38.138273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c561664d1a74586b438804856931412'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:38.151749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c561664d1a74586b438804856931412'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:38.184128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 897/1785] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:37 2022] DELETE /identity/v3/users/6c561664d1a74586b438804856931412 => generated 0 bytes in 821 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:38.185907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:38.189564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 889/1786] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:37 2022] POST /identity/v3/users => generated 311 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:38.191211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7bda3-e0a2-448e-a60a-d63321ee88fd admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:38.194025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-51d7bda3-e0a2-448e-a60a-d63321ee88fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:38.196468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:38.228110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7bda3-e0a2-448e-a60a-d63321ee88fd admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:38.346197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16c14f-d258-44bd-ac9c-a160bc49de64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:38.347430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16c14f-d258-44bd-ac9c-a160bc49de64 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:38.349859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16c14f-d258-44bd-ac9c-a160bc49de64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:38.350754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16c14f-d258-44bd-ac9c-a160bc49de64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:38.364967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16c14f-d258-44bd-ac9c-a160bc49de64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:38.371884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16c14f-d258-44bd-ac9c-a160bc49de64 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:38.428252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 890/1787] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:38.442146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16c14f-d258-44bd-ac9c-a160bc49de64 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:38.480302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47868050-c938-4538-8f23-e777930f96d3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:38.483133 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16c14f-d258-44bd-ac9c-a160bc49de64 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:38.505006 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43528260-d946-4217-a280-40b98a7b8eda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:38.511258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43528260-d946-4217-a280-40b98a7b8eda service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:38.512799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43528260-d946-4217-a280-40b98a7b8eda service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:38.515147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4d9ff-ac33-4032-b972-1b3345367832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:38.515839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43528260-d946-4217-a280-40b98a7b8eda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:38.517010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4d9ff-ac33-4032-b972-1b3345367832 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:38.517303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4d9ff-ac33-4032-b972-1b3345367832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:38.523011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4d9ff-ac33-4032-b972-1b3345367832 admin admin] PATH_INFO: `/v3/projects/ccf7ef480a5c41988752a408ec363341/users/aef0d315d3454710b98cca15f189efc4/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:38.527721 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-43528260-d946-4217-a280-40b98a7b8eda service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:38.563363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4d9ff-ac33-4032-b972-1b3345367832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf7ef480a5c41988752a408ec363341, user_id=aef0d315d3454710b98cca15f189efc4, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:38.568503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:38.570510 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4d9ff-ac33-4032-b972-1b3345367832 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:38.672172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43528260-d946-4217-a280-40b98a7b8eda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:38.674043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43528260-d946-4217-a280-40b98a7b8eda service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:38.685411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 898/1788] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:29:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:29:38.707087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 891/1789] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:38 2022] PUT /identity/v3/projects/ccf7ef480a5c41988752a408ec363341/users/aef0d315d3454710b98cca15f189efc4/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:29:38.711740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43528260-d946-4217-a280-40b98a7b8eda service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:38.791606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43528260-d946-4217-a280-40b98a7b8eda service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:38.904270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c860986-753c-4bed-9d16-cd429492bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:38.907090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c860986-753c-4bed-9d16-cd429492bc4e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:38.907450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c860986-753c-4bed-9d16-cd429492bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:38.907960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c860986-753c-4bed-9d16-cd429492bc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:38.909080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c860986-753c-4bed-9d16-cd429492bc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:38.910963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c860986-753c-4bed-9d16-cd429492bc4e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:38.924922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 899/1790] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:38.930279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4d9ff-ac33-4032-b972-1b3345367832 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:38.959639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4d9ff-ac33-4032-b972-1b3345367832 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:38.971067 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112e23e-5005-46f8-ac9a-15fc1b575621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:38.976510 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112e23e-5005-46f8-ac9a-15fc1b575621 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:38.977188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112e23e-5005-46f8-ac9a-15fc1b575621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:38.977658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112e23e-5005-46f8-ac9a-15fc1b575621 admin admin] PATH_INFO: `/v3/projects/ccf7ef480a5c41988752a408ec363341/users/aef0d315d3454710b98cca15f189efc4/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:38.982129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112e23e-5005-46f8-ac9a-15fc1b575621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf7ef480a5c41988752a408ec363341, user_id=aef0d315d3454710b98cca15f189efc4, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:38.985463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112e23e-5005-46f8-ac9a-15fc1b575621 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:39.012740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 892/1791] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:38 2022] PUT /identity/v3/projects/ccf7ef480a5c41988752a408ec363341/users/aef0d315d3454710b98cca15f189efc4/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:39.024285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74a3b0-ed90-43fa-a11c-c020e785a453 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:39.025305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74a3b0-ed90-43fa-a11c-c020e785a453 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:39.025701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74a3b0-ed90-43fa-a11c-c020e785a453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:39.095030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6b74a3b0-ed90-43fa-a11c-c020e785a453 None None] MFA Rules not processed for user `aef0d315d3454710b98cca15f189efc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:39.099376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6b74a3b0-ed90-43fa-a11c-c020e785a453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:39.300259 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112e23e-5005-46f8-ac9a-15fc1b575621 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:39.304668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5112e23e-5005-46f8-ac9a-15fc1b575621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:39.347393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 900/1792] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:39.361753 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74a3b0-ed90-43fa-a11c-c020e785a453 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:39.363483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6b74a3b0-ed90-43fa-a11c-c020e785a453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:39.484779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112e23e-5005-46f8-ac9a-15fc1b575621 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:39.523147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2af4c9-a21d-4031-8b89-19808f8f2950 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:39.525063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2af4c9-a21d-4031-8b89-19808f8f2950 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:39.525519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2af4c9-a21d-4031-8b89-19808f8f2950 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:39.525847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2af4c9-a21d-4031-8b89-19808f8f2950 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:39.528162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-da2af4c9-a21d-4031-8b89-19808f8f2950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:39.631238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74a3b0-ed90-43fa-a11c-c020e785a453 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:39.639068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d083b-d985-4ec9-b779-c80c2eb23f87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:39.640374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d083b-d985-4ec9-b779-c80c2eb23f87 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:39.641546 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d083b-d985-4ec9-b779-c80c2eb23f87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:39.642860 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d083b-d985-4ec9-b779-c80c2eb23f87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:39.718522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d083b-d985-4ec9-b779-c80c2eb23f87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:39.727893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d083b-d985-4ec9-b779-c80c2eb23f87 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:39.761701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2af4c9-a21d-4031-8b89-19808f8f2950 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:39.763202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2af4c9-a21d-4031-8b89-19808f8f2950 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:39.767390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 901/1793] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:39 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:29:39.784244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c195461-bbd9-43ad-970e-475946ab88f8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:39.785773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c195461-bbd9-43ad-970e-475946ab88f8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:39.788248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 893/1794] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:39.788485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c195461-bbd9-43ad-970e-475946ab88f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:39.903197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5c195461-bbd9-43ad-970e-475946ab88f8 None None] MFA Rules not processed for user `9a89a2e6aeb4495a84a61bd82b02bfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:39.904717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5c195461-bbd9-43ad-970e-475946ab88f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:40.044014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 902/1795] 10.222.0.58 () {58 vars in 1015 bytes} [Mon Sep 5 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:40.159618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2af4c9-a21d-4031-8b89-19808f8f2950 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:40.163160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-da2af4c9-a21d-4031-8b89-19808f8f2950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:40.177379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c195461-bbd9-43ad-970e-475946ab88f8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:40.278059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c195461-bbd9-43ad-970e-475946ab88f8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:40.278059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0225868-4a12-4ac4-8205-e2603956195b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:40.279318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2af4c9-a21d-4031-8b89-19808f8f2950 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:40.279622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0225868-4a12-4ac4-8205-e2603956195b service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:40.279622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0225868-4a12-4ac4-8205-e2603956195b service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:40.279622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0225868-4a12-4ac4-8205-e2603956195b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:40.279874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237dfde-7a76-4c3e-8ca4-dbeccf89fbbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:40.279874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237dfde-7a76-4c3e-8ca4-dbeccf89fbbc service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:40.279874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237dfde-7a76-4c3e-8ca4-dbeccf89fbbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:40.279874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237dfde-7a76-4c3e-8ca4-dbeccf89fbbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:40.280366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0225868-4a12-4ac4-8205-e2603956195b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:40.280366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0225868-4a12-4ac4-8205-e2603956195b service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:40.280523 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237dfde-7a76-4c3e-8ca4-dbeccf89fbbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:40.284289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237dfde-7a76-4c3e-8ca4-dbeccf89fbbc service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:40.302492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 903/1796] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:40.316691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 894/1797] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:40.747044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0225868-4a12-4ac4-8205-e2603956195b service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:40.750896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b0225868-4a12-4ac4-8205-e2603956195b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:40.984965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237dfde-7a76-4c3e-8ca4-dbeccf89fbbc service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:40.999274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0225868-4a12-4ac4-8205-e2603956195b service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:41.008967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237dfde-7a76-4c3e-8ca4-dbeccf89fbbc service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:41.013083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:41.013646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:41.014909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:41.015194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] PATH_INFO: `/v3/projects/6e3209e42263420c81a04c878809ad9f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:41.018009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3209e42263420c81a04c878809ad9f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:41.020020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208b8e9-03f2-4b1b-8526-562870e15a15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:41.022296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:41.023093 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208b8e9-03f2-4b1b-8526-562870e15a15 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:41.023490 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208b8e9-03f2-4b1b-8526-562870e15a15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:41.026104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208b8e9-03f2-4b1b-8526-562870e15a15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:41.030901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f208b8e9-03f2-4b1b-8526-562870e15a15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:41.203277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:41.206482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208b8e9-03f2-4b1b-8526-562870e15a15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:41.209251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208b8e9-03f2-4b1b-8526-562870e15a15 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:41.237272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 895/1798] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:40 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:41.308707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3209e42263420c81a04c878809ad9f', 'request_id': 'req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:41.317924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3209e42263420c81a04c878809ad9f', 'request_id': 'req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:41.478394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3209e42263420c81a04c878809ad9f', 'request_id': 'req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:41.497767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] The token cache is being invalidate because project 6e3209e42263420c81a04c878809ad9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:41.498156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:41.507191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-dd9bd8ea-cdc7-4ea0-bd47-53f4fe3b737c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:41.514425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 904/1799] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:40 2022] DELETE /identity/v3/projects/6e3209e42263420c81a04c878809ad9f => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:41.515712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa80153-c7bf-49e7-abb0-a24aea08ffcf None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:41.516198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa80153-c7bf-49e7-abb0-a24aea08ffcf None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:41.516293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa80153-c7bf-49e7-abb0-a24aea08ffcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:41.569929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6fa80153-c7bf-49e7-abb0-a24aea08ffcf None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:41.572220 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6fa80153-c7bf-49e7-abb0-a24aea08ffcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:41.664472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:41.839872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 896/1800] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:29:41.841893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69eb6-99bb-46eb-ba0b-1c9967e4ef28 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:41.842436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69eb6-99bb-46eb-ba0b-1c9967e4ef28 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:41.842668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69eb6-99bb-46eb-ba0b-1c9967e4ef28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:41.898507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-dab69eb6-99bb-46eb-ba0b-1c9967e4ef28 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:41.900012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dab69eb6-99bb-46eb-ba0b-1c9967e4ef28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:41.925418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 905/1801] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:41.927386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa80153-c7bf-49e7-abb0-a24aea08ffcf None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:41.942935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa80153-c7bf-49e7-abb0-a24aea08ffcf None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:41.949149 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc63e9-8e89-4171-8c83-e17dd3a0ffa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:41.950611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc63e9-8e89-4171-8c83-e17dd3a0ffa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:41.951366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc63e9-8e89-4171-8c83-e17dd3a0ffa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:41.951874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc63e9-8e89-4171-8c83-e17dd3a0ffa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:41.953083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc63e9-8e89-4171-8c83-e17dd3a0ffa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:41.953574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc63e9-8e89-4171-8c83-e17dd3a0ffa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:41.956077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc63e9-8e89-4171-8c83-e17dd3a0ffa1 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:41.989110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 897/1802] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:41.994742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69eb6-99bb-46eb-ba0b-1c9967e4ef28 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:42.012151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69eb6-99bb-46eb-ba0b-1c9967e4ef28 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:42.028457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533e4c1-1be8-4f92-9f47-a36be455db1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:42.028457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533e4c1-1be8-4f92-9f47-a36be455db1a admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:42.028457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533e4c1-1be8-4f92-9f47-a36be455db1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:42.028457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533e4c1-1be8-4f92-9f47-a36be455db1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:42.028793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533e4c1-1be8-4f92-9f47-a36be455db1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:42.035341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533e4c1-1be8-4f92-9f47-a36be455db1a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:42.313671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 906/1803] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:41 2022] POST /identity/v3/projects => generated 365 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:42.318575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc63e9-8e89-4171-8c83-e17dd3a0ffa1 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:42.368218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc63e9-8e89-4171-8c83-e17dd3a0ffa1 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:42.520163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3c9ee-77c2-4724-90d2-572765a0df01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:42.521465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3c9ee-77c2-4724-90d2-572765a0df01 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:42.533862 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3c9ee-77c2-4724-90d2-572765a0df01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:42.534301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3c9ee-77c2-4724-90d2-572765a0df01 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:42.535220 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3c9ee-77c2-4724-90d2-572765a0df01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:42.537613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3c9ee-77c2-4724-90d2-572765a0df01 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:42.731972 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 898/1804] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:42 2022] POST /identity/v3/users => generated 283 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:42.733264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533e4c1-1be8-4f92-9f47-a36be455db1a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:42.751782 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533e4c1-1be8-4f92-9f47-a36be455db1a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:42.758659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f23208-3b78-43de-b414-9da52c704687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:42.759517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f23208-3b78-43de-b414-9da52c704687 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:42.759986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f23208-3b78-43de-b414-9da52c704687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:42.760488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f23208-3b78-43de-b414-9da52c704687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:42.761486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f23208-3b78-43de-b414-9da52c704687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:42.768883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f23208-3b78-43de-b414-9da52c704687 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:42.806712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 907/1805] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:42.809723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3c9ee-77c2-4724-90d2-572765a0df01 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:42.823754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3c9ee-77c2-4724-90d2-572765a0df01 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:42.843125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8970-3f09-49ce-bcf6-f17cf43b4f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:42.843923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8970-3f09-49ce-bcf6-f17cf43b4f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:42.844754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8970-3f09-49ce-bcf6-f17cf43b4f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:42.845315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8970-3f09-49ce-bcf6-f17cf43b4f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:42.847973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a8970-3f09-49ce-bcf6-f17cf43b4f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:42.854850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a8970-3f09-49ce-bcf6-f17cf43b4f92 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:42.881476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 899/1806] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:42.892164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f23208-3b78-43de-b414-9da52c704687 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:42.913671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f23208-3b78-43de-b414-9da52c704687 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:42.932890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a9330-46b2-4788-8193-c7412e5fcf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:42.933771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a9330-46b2-4788-8193-c7412e5fcf9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:42.934205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a9330-46b2-4788-8193-c7412e5fcf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:42.934621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a9330-46b2-4788-8193-c7412e5fcf9d admin admin] PATH_INFO: `/v3/projects/ba12976666de475194a11697e4183b97/users/2fd7fb2bd0f043ec9444c2c73c700513/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:42.990729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a9330-46b2-4788-8193-c7412e5fcf9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba12976666de475194a11697e4183b97, user_id=2fd7fb2bd0f043ec9444c2c73c700513, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:42.992075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a9330-46b2-4788-8193-c7412e5fcf9d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:43.125790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 908/1807] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:42 2022] PUT /identity/v3/projects/ba12976666de475194a11697e4183b97/users/2fd7fb2bd0f043ec9444c2c73c700513/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:43.142658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aeb6c-22dc-4b09-8fa1-6bcf0b1ddeba None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:43.143031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aeb6c-22dc-4b09-8fa1-6bcf0b1ddeba None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:43.143382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aeb6c-22dc-4b09-8fa1-6bcf0b1ddeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:43.217097 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-177aeb6c-22dc-4b09-8fa1-6bcf0b1ddeba None None] MFA Rules not processed for user `2fd7fb2bd0f043ec9444c2c73c700513`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:43.224437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-177aeb6c-22dc-4b09-8fa1-6bcf0b1ddeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:43.418370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 900/1808] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:43.423915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a9330-46b2-4788-8193-c7412e5fcf9d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:43.586946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a9330-46b2-4788-8193-c7412e5fcf9d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:43.601450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66403c3-675d-447e-9d68-353c0f814d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:43.602184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66403c3-675d-447e-9d68-353c0f814d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:43.603775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66403c3-675d-447e-9d68-353c0f814d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:43.606196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66403c3-675d-447e-9d68-353c0f814d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:43.606719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66403c3-675d-447e-9d68-353c0f814d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:43.612335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66403c3-675d-447e-9d68-353c0f814d0d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:43.812588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 909/1809] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:43 2022] POST /identity/v3/projects => generated 363 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:43.817069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aeb6c-22dc-4b09-8fa1-6bcf0b1ddeba None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:43.836577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aeb6c-22dc-4b09-8fa1-6bcf0b1ddeba None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:44.039235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5dc5a-483b-448d-aaea-5089850a2f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:44.040321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5dc5a-483b-448d-aaea-5089850a2f98 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:44.048932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5dc5a-483b-448d-aaea-5089850a2f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:44.054974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5dc5a-483b-448d-aaea-5089850a2f98 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:44.054974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5dc5a-483b-448d-aaea-5089850a2f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:44.059114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5dc5a-483b-448d-aaea-5089850a2f98 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:44.247771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 901/1810] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:43 2022] POST /identity/v3/users => generated 288 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:44.251777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66403c3-675d-447e-9d68-353c0f814d0d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:44.285182 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66403c3-675d-447e-9d68-353c0f814d0d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:44.303419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b9cfe-e7ab-4522-8dfc-5788fda3db42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:44.304553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b9cfe-e7ab-4522-8dfc-5788fda3db42 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:44.305045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b9cfe-e7ab-4522-8dfc-5788fda3db42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:44.305551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b9cfe-e7ab-4522-8dfc-5788fda3db42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:44.306512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b9cfe-e7ab-4522-8dfc-5788fda3db42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:44.309559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b9cfe-e7ab-4522-8dfc-5788fda3db42 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:44.352956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 910/1811] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:44.358361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5dc5a-483b-448d-aaea-5089850a2f98 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:44.380181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:44.386035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5dc5a-483b-448d-aaea-5089850a2f98 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:44.402483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b58212-435d-4ad5-bee8-87f011807ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:44.404107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b58212-435d-4ad5-bee8-87f011807ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:44.404991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b58212-435d-4ad5-bee8-87f011807ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:44.405657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b58212-435d-4ad5-bee8-87f011807ea5 admin admin] PATH_INFO: `/v3/projects/7a9ebe81aa324299aa386a2e2ffbf6e8/users/7656c3c3ebef4baa90c541695dbef060/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:44.444491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b58212-435d-4ad5-bee8-87f011807ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9ebe81aa324299aa386a2e2ffbf6e8, user_id=7656c3c3ebef4baa90c541695dbef060, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:44.452719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b58212-435d-4ad5-bee8-87f011807ea5 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:44.610958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 902/1812] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:44 2022] PUT /identity/v3/projects/7a9ebe81aa324299aa386a2e2ffbf6e8/users/7656c3c3ebef4baa90c541695dbef060/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:44.617228 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b9cfe-e7ab-4522-8dfc-5788fda3db42 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:44.638816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b9cfe-e7ab-4522-8dfc-5788fda3db42 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:44.865287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d76f7-da9b-479f-b75f-f076c24da4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:44.866193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d76f7-da9b-479f-b75f-f076c24da4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:44.866656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d76f7-da9b-479f-b75f-f076c24da4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:44.867067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d76f7-da9b-479f-b75f-f076c24da4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:44.867911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d76f7-da9b-479f-b75f-f076c24da4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:44.871409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d76f7-da9b-479f-b75f-f076c24da4c3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:44.917246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:44.919409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 911/1813] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:44.923171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b58212-435d-4ad5-bee8-87f011807ea5 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:44.944888 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b58212-435d-4ad5-bee8-87f011807ea5 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:44.954110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0c2c-09f1-42fc-9fb7-5ce869ad3062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:44.955855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0c2c-09f1-42fc-9fb7-5ce869ad3062 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:44.956350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0c2c-09f1-42fc-9fb7-5ce869ad3062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:44.956785 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0c2c-09f1-42fc-9fb7-5ce869ad3062 admin admin] PATH_INFO: `/v3/projects/7a9ebe81aa324299aa386a2e2ffbf6e8/users/7656c3c3ebef4baa90c541695dbef060/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:44.961565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb0c2c-09f1-42fc-9fb7-5ce869ad3062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9ebe81aa324299aa386a2e2ffbf6e8, user_id=7656c3c3ebef4baa90c541695dbef060, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:44.965109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb0c2c-09f1-42fc-9fb7-5ce869ad3062 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:45.013568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 903/1814] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:44 2022] PUT /identity/v3/projects/7a9ebe81aa324299aa386a2e2ffbf6e8/users/7656c3c3ebef4baa90c541695dbef060/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:45.023609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e049b7-432a-4ba2-ab1d-b16f4dab4e54 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:45.024274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e049b7-432a-4ba2-ab1d-b16f4dab4e54 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:45.024864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e049b7-432a-4ba2-ab1d-b16f4dab4e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:45.176424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-64e049b7-432a-4ba2-ab1d-b16f4dab4e54 None None] MFA Rules not processed for user `7656c3c3ebef4baa90c541695dbef060`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:45.177691 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-64e049b7-432a-4ba2-ab1d-b16f4dab4e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:45.364024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 912/1815] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:45.377248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0c2c-09f1-42fc-9fb7-5ce869ad3062 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:45.379301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3bbb0c2c-09f1-42fc-9fb7-5ce869ad3062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:45.569910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0c2c-09f1-42fc-9fb7-5ce869ad3062 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:45.576086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52f5f6-f2a9-4c2c-98b5-6caf3398b57d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:45.577468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52f5f6-f2a9-4c2c-98b5-6caf3398b57d service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:45.578105 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52f5f6-f2a9-4c2c-98b5-6caf3398b57d service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:45.578995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52f5f6-f2a9-4c2c-98b5-6caf3398b57d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:45.599646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52f5f6-f2a9-4c2c-98b5-6caf3398b57d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:45.604846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52f5f6-f2a9-4c2c-98b5-6caf3398b57d service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:45.641971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 904/1816] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:29:45.667197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2714dbf-b24e-4af4-92ed-499a5986ef89 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:45.667197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2714dbf-b24e-4af4-92ed-499a5986ef89 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:45.667197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2714dbf-b24e-4af4-92ed-499a5986ef89 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:45.667197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 913/1817] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:29:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:29:45.709232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e01b7d-b93e-43af-8cbd-7bfdff7c1254 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:45.709526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e01b7d-b93e-43af-8cbd-7bfdff7c1254 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:45.709844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e01b7d-b93e-43af-8cbd-7bfdff7c1254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:45.740181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e6e01b7d-b93e-43af-8cbd-7bfdff7c1254 None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:45.740521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e6e01b7d-b93e-43af-8cbd-7bfdff7c1254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:45.937169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 905/1818] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:29:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:29:46.003151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2714dbf-b24e-4af4-92ed-499a5986ef89 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:46.007892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c2714dbf-b24e-4af4-92ed-499a5986ef89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:46.028353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2714dbf-b24e-4af4-92ed-499a5986ef89 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:46.034970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69c12-2a5b-42ba-9bbb-bdc40fa0dc16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:46.036131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69c12-2a5b-42ba-9bbb-bdc40fa0dc16 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:46.036553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69c12-2a5b-42ba-9bbb-bdc40fa0dc16 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:46.036962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69c12-2a5b-42ba-9bbb-bdc40fa0dc16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:46.058022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b69c12-2a5b-42ba-9bbb-bdc40fa0dc16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:46.059844 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b69c12-2a5b-42ba-9bbb-bdc40fa0dc16 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:46.073089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991a0c1-057b-430f-b438-ba4fc44f4de7 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:46.073533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991a0c1-057b-430f-b438-ba4fc44f4de7 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:46.073684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991a0c1-057b-430f-b438-ba4fc44f4de7 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:46.076711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 906/1819] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:29:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:29:46.083823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 914/1820] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:46.102473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c81897-4357-4719-b251-e3d7ed2136cc None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:46.103126 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c81897-4357-4719-b251-e3d7ed2136cc None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:46.103737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c81897-4357-4719-b251-e3d7ed2136cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:46.134241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e4c81897-4357-4719-b251-e3d7ed2136cc None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:46.136696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e4c81897-4357-4719-b251-e3d7ed2136cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:46.150675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 907/1821] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:46.187287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69c12-2a5b-42ba-9bbb-bdc40fa0dc16 service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:46.195400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69c12-2a5b-42ba-9bbb-bdc40fa0dc16 service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:46.202953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3a439-0c2f-4971-8872-d0f195574de2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:46.203590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a439-0c2f-4971-8872-d0f195574de2 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:46.204049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a439-0c2f-4971-8872-d0f195574de2 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:46.204465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a439-0c2f-4971-8872-d0f195574de2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:46.218170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3a439-0c2f-4971-8872-d0f195574de2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:46.219808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3a439-0c2f-4971-8872-d0f195574de2 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:46.231852 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 915/1822] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:46.437587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:46.455379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c81897-4357-4719-b251-e3d7ed2136cc None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:46.456982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e4c81897-4357-4719-b251-e3d7ed2136cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:46.471463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c81897-4357-4719-b251-e3d7ed2136cc None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:46.476993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8443cc-04ab-49ae-aec5-2049365c099a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:46.477895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8443cc-04ab-49ae-aec5-2049365c099a service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:46.478920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8443cc-04ab-49ae-aec5-2049365c099a service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:46.478920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8443cc-04ab-49ae-aec5-2049365c099a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:46.546641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8443cc-04ab-49ae-aec5-2049365c099a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:46.548424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8443cc-04ab-49ae-aec5-2049365c099a service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:46.558125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 908/1823] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:46.678114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3a439-0c2f-4971-8872-d0f195574de2 service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:46.679705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e1b3a439-0c2f-4971-8872-d0f195574de2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:46.803908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3a439-0c2f-4971-8872-d0f195574de2 service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:46.809010 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:46.809673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:46.810096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:46.818817 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] PATH_INFO: `/v3/users/f4a8481413114913a41e9db7221812a1` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:46.818817 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4a8481413114913a41e9db7221812a1)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:46.818817 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:47.122665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4a8481413114913a41e9db7221812a1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:47.123774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4a8481413114913a41e9db7221812a1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:47.314384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4a8481413114913a41e9db7221812a1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:47.328346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4a8481413114913a41e9db7221812a1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:47.329052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4a8481413114913a41e9db7221812a1'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:47.341610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 916/1824] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:46 2022] DELETE /identity/v3/users/f4a8481413114913a41e9db7221812a1 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:47.346243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262dc7-7a03-40b6-873e-773c14711541 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:47.346749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262dc7-7a03-40b6-873e-773c14711541 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:47.346838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262dc7-7a03-40b6-873e-773c14711541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:47.369783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8c262dc7-7a03-40b6-873e-773c14711541 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:47.371135 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8c262dc7-7a03-40b6-873e-773c14711541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:47.435687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 909/1825] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:47.446067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:47.447579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:47.505959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba186cf-0aef-404f-ab40-eedbe9515b99 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:47.509465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667b759-c559-491a-aa8c-49a71dfd6295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:47.510073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b759-c559-491a-aa8c-49a71dfd6295 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:47.510346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b759-c559-491a-aa8c-49a71dfd6295 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:47.510626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b759-c559-491a-aa8c-49a71dfd6295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:47.519597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667b759-c559-491a-aa8c-49a71dfd6295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:47.521044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667b759-c559-491a-aa8c-49a71dfd6295 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:47.530418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 917/1826] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:29:47.611887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:48.010865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c262dc7-7a03-40b6-873e-773c14711541 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:48.012369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8c262dc7-7a03-40b6-873e-773c14711541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:48.024658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c262dc7-7a03-40b6-873e-773c14711541 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:48.030452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:48.031266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:48.031851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:48.032271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] PATH_INFO: `/v3/projects/b03b6414ccce40da8111fa7dba1b02c5` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:48.033856 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b03b6414ccce40da8111fa7dba1b02c5)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:48.048781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:48.351856 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b03b6414ccce40da8111fa7dba1b02c5', 'request_id': 'req-1024640f-3ef5-4797-8e48-d7fcae1017ba'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:48.363115 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b03b6414ccce40da8111fa7dba1b02c5', 'request_id': 'req-1024640f-3ef5-4797-8e48-d7fcae1017ba'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:48.546763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b03b6414ccce40da8111fa7dba1b02c5', 'request_id': 'req-1024640f-3ef5-4797-8e48-d7fcae1017ba'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:48.566837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] The token cache is being invalidate because project b03b6414ccce40da8111fa7dba1b02c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:48.567875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:48.568783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:48.573178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 910/1827] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:48 2022] DELETE /identity/v3/projects/b03b6414ccce40da8111fa7dba1b02c5 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:48.574385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667b759-c559-491a-aa8c-49a71dfd6295 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:48.588932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667b759-c559-491a-aa8c-49a71dfd6295 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:48.687952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:48.688842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:48.689380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:48.689826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] PATH_INFO: `/v3/users/81599ae092de4b9aa8cad5a818d52890` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:48.691389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=81599ae092de4b9aa8cad5a818d52890)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:48.694192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:48.927511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81599ae092de4b9aa8cad5a818d52890'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:48.928817 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81599ae092de4b9aa8cad5a818d52890'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:49.017353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81599ae092de4b9aa8cad5a818d52890'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:49.037492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81599ae092de4b9aa8cad5a818d52890'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:49.038502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81599ae092de4b9aa8cad5a818d52890'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:49.059310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 918/1828] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:48 2022] DELETE /identity/v3/users/81599ae092de4b9aa8cad5a818d52890 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:49.313624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:49.628935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:49.914592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:49.916328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:49.938675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024640f-3ef5-4797-8e48-d7fcae1017ba admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:50.033604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:50.034551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:50.034979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:50.035426 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] PATH_INFO: `/v3/projects/4ead8ea0c6324200b8f5ca4389a168f3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:50.037009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ead8ea0c6324200b8f5ca4389a168f3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:50.039496 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:50.187424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ead8ea0c6324200b8f5ca4389a168f3', 'request_id': 'req-ef447f9f-bc8e-4849-a516-fcabfe520574'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:50.199389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ead8ea0c6324200b8f5ca4389a168f3', 'request_id': 'req-ef447f9f-bc8e-4849-a516-fcabfe520574'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:50.366037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ead8ea0c6324200b8f5ca4389a168f3', 'request_id': 'req-ef447f9f-bc8e-4849-a516-fcabfe520574'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:50.389885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] The token cache is being invalidate because project 4ead8ea0c6324200b8f5ca4389a168f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:50.390315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:50.391244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:50.393210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 911/1829] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:49 2022] DELETE /identity/v3/projects/4ead8ea0c6324200b8f5ca4389a168f3 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:50.401686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:50.405040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:50.419133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7e659-d58a-429e-8f14-a36c871a77cf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:50.493447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:50.500142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:50.500744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:50.501205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] PATH_INFO: `/v3/users/c2d9139857024eaab3778469c5c9a708` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:50.502675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d9139857024eaab3778469c5c9a708)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:50.505797 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:50.782978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d9139857024eaab3778469c5c9a708'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:50.784690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d9139857024eaab3778469c5c9a708'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:50.917187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d9139857024eaab3778469c5c9a708'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:50.936743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d9139857024eaab3778469c5c9a708'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:50.938000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-09f7f9fd-15b7-45b3-9a3a-8fb77cfa20f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d9139857024eaab3778469c5c9a708'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:50.956333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 919/1830] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:50 2022] DELETE /identity/v3/users/c2d9139857024eaab3778469c5c9a708 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:52.531731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:52.889039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:52.890529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:52.917313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef447f9f-bc8e-4849-a516-fcabfe520574 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:53.033814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:53.034466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:53.034749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:53.035023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] PATH_INFO: `/v3/projects/63afe0aff49d4936ae83a76c56ead467` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:53.036841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63afe0aff49d4936ae83a76c56ead467)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:53.038786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:53.320721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63afe0aff49d4936ae83a76c56ead467', 'request_id': 'req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:53.326694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63afe0aff49d4936ae83a76c56ead467', 'request_id': 'req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:53.453924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63afe0aff49d4936ae83a76c56ead467', 'request_id': 'req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:53.466475 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] The token cache is being invalidate because project 63afe0aff49d4936ae83a76c56ead467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:53.474927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:53.474927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c089b10-b48e-4fb9-8d52-ecaf4eb5f915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:53.474927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 912/1831] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:52 2022] DELETE /identity/v3/projects/63afe0aff49d4936ae83a76c56ead467 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:53.476559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683b1f-41f5-4252-a6d6-014846d7048e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:53.477044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683b1f-41f5-4252-a6d6-014846d7048e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:53.477138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683b1f-41f5-4252-a6d6-014846d7048e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:53.497099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2d683b1f-41f5-4252-a6d6-014846d7048e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:53.498375 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2d683b1f-41f5-4252-a6d6-014846d7048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:53.558400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 920/1832] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:53.566519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fe6f0-7563-4865-911f-530de6a216b4 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:53.567658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fe6f0-7563-4865-911f-530de6a216b4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:53.567658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fe6f0-7563-4865-911f-530de6a216b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:53.587673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d16fe6f0-7563-4865-911f-530de6a216b4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:53.588890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d16fe6f0-7563-4865-911f-530de6a216b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:53.597201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 913/1833] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:53.601535 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d683b1f-41f5-4252-a6d6-014846d7048e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:53.611868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d683b1f-41f5-4252-a6d6-014846d7048e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:53.616725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a15a0c-bcfb-4b7a-a00f-fc34b7ea813c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:53.617343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a15a0c-bcfb-4b7a-a00f-fc34b7ea813c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:53.617620 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a15a0c-bcfb-4b7a-a00f-fc34b7ea813c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:53.617888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a15a0c-bcfb-4b7a-a00f-fc34b7ea813c admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:53.618547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a15a0c-bcfb-4b7a-a00f-fc34b7ea813c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:53.618854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a15a0c-bcfb-4b7a-a00f-fc34b7ea813c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:53.620483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a15a0c-bcfb-4b7a-a00f-fc34b7ea813c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:53.636068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 921/1834] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:29:53.639941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fe6f0-7563-4865-911f-530de6a216b4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:53.650574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fe6f0-7563-4865-911f-530de6a216b4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:53.656299 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da66858-0763-4b9b-8909-ae0d94b5d372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:53.662011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da66858-0763-4b9b-8909-ae0d94b5d372 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:53.662011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da66858-0763-4b9b-8909-ae0d94b5d372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:53.662011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da66858-0763-4b9b-8909-ae0d94b5d372 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:53.662011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da66858-0763-4b9b-8909-ae0d94b5d372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:53.662011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da66858-0763-4b9b-8909-ae0d94b5d372 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:53.827704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 914/1835] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:53 2022] POST /identity/v3/projects => generated 401 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:53.831779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a15a0c-bcfb-4b7a-a00f-fc34b7ea813c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:53.849643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a15a0c-bcfb-4b7a-a00f-fc34b7ea813c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:53.975568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a26033-2ef9-4a5b-83a1-179d9fd831fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:53.976068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a26033-2ef9-4a5b-83a1-179d9fd831fb admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:53.976467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a26033-2ef9-4a5b-83a1-179d9fd831fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:53.976843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a26033-2ef9-4a5b-83a1-179d9fd831fb admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:53.977717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a26033-2ef9-4a5b-83a1-179d9fd831fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:53.980225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a26033-2ef9-4a5b-83a1-179d9fd831fb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:54.030435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da66858-0763-4b9b-8909-ae0d94b5d372 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:54.031825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6da66858-0763-4b9b-8909-ae0d94b5d372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:54.068196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da66858-0763-4b9b-8909-ae0d94b5d372 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:54.071804 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:54.072713 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:54.073211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:54.073670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] PATH_INFO: `/v3/users/c96ec793badf41b4b6c9c39807f848c7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:54.075448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96ec793badf41b4b6c9c39807f848c7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:54.078132 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:54.156353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 922/1836] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:53 2022] POST /identity/v3/users => generated 301 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:54.157865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a26033-2ef9-4a5b-83a1-179d9fd831fb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:54.168570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a26033-2ef9-4a5b-83a1-179d9fd831fb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:54.173643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ddb45-11f6-40a6-9529-c08212efa778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:54.174249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ddb45-11f6-40a6-9529-c08212efa778 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:54.174626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ddb45-11f6-40a6-9529-c08212efa778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:54.174785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ddb45-11f6-40a6-9529-c08212efa778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:54.175428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ddb45-11f6-40a6-9529-c08212efa778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:54.177790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ddb45-11f6-40a6-9529-c08212efa778 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:54.188846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 923/1837] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:54.193549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ddb45-11f6-40a6-9529-c08212efa778 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:54.203448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ddb45-11f6-40a6-9529-c08212efa778 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:54.208652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28842057-57e8-4c43-8642-396bdc8eeb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:54.209371 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28842057-57e8-4c43-8642-396bdc8eeb4d admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:54.209713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28842057-57e8-4c43-8642-396bdc8eeb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:54.210049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28842057-57e8-4c43-8642-396bdc8eeb4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:54.210785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28842057-57e8-4c43-8642-396bdc8eeb4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:54.212535 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28842057-57e8-4c43-8642-396bdc8eeb4d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:54.223235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 924/1838] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:29:54.228139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28842057-57e8-4c43-8642-396bdc8eeb4d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:54.238022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28842057-57e8-4c43-8642-396bdc8eeb4d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:54.243274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d69ac-4bc2-40c1-a53e-00959e4f1a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:54.243995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d69ac-4bc2-40c1-a53e-00959e4f1a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:54.244305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d69ac-4bc2-40c1-a53e-00959e4f1a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:54.244568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d69ac-4bc2-40c1-a53e-00959e4f1a01 admin admin] PATH_INFO: `/v3/projects/6c414ade18ea4ad2a562ae77802b472f/users/db49e7b50f304c94be3a4ad96598bcf2/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:54.267594 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d69ac-4bc2-40c1-a53e-00959e4f1a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c414ade18ea4ad2a562ae77802b472f, user_id=db49e7b50f304c94be3a4ad96598bcf2, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:54.267594 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d69ac-4bc2-40c1-a53e-00959e4f1a01 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:54.359998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96ec793badf41b4b6c9c39807f848c7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:54.360679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96ec793badf41b4b6c9c39807f848c7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:54.471667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96ec793badf41b4b6c9c39807f848c7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:54.478531 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 925/1839] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:54 2022] PUT /identity/v3/projects/6c414ade18ea4ad2a562ae77802b472f/users/db49e7b50f304c94be3a4ad96598bcf2/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:29:54.487499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96ec793badf41b4b6c9c39807f848c7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:54.490338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12805b-8677-4d91-9ffb-db0a6c009c01 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:54.490609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12805b-8677-4d91-9ffb-db0a6c009c01 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:54.490877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12805b-8677-4d91-9ffb-db0a6c009c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:54.494134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-364a0b02-ed4e-4455-bcdf-9a5ce7020e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96ec793badf41b4b6c9c39807f848c7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:54.510360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 915/1840] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:54 2022] DELETE /identity/v3/users/c96ec793badf41b4b6c9c39807f848c7 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:29:54.514744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2799b5-f071-4c88-a6eb-bacf33f35db8 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:54.515207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2799b5-f071-4c88-a6eb-bacf33f35db8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:54.515468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2799b5-f071-4c88-a6eb-bacf33f35db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:54.531002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5b12805b-8677-4d91-9ffb-db0a6c009c01 None None] MFA Rules not processed for user `db49e7b50f304c94be3a4ad96598bcf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:54.531002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5b12805b-8677-4d91-9ffb-db0a6c009c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:54.542128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-fe2799b5-f071-4c88-a6eb-bacf33f35db8 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:54.543384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fe2799b5-f071-4c88-a6eb-bacf33f35db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:54.574687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:54.596414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 926/1841] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:54.601785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12805b-8677-4d91-9ffb-db0a6c009c01 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:54.603093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5b12805b-8677-4d91-9ffb-db0a6c009c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:54.607658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 916/1842] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:29:54.617942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12805b-8677-4d91-9ffb-db0a6c009c01 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:54.618108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2799b5-f071-4c88-a6eb-bacf33f35db8 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:54.619440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fe2799b5-f071-4c88-a6eb-bacf33f35db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:54.623381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a1e44-50f8-4b35-b189-8d815d400766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:54.624047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a1e44-50f8-4b35-b189-8d815d400766 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:54.624317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a1e44-50f8-4b35-b189-8d815d400766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:54.624583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a1e44-50f8-4b35-b189-8d815d400766 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:54.625206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a1e44-50f8-4b35-b189-8d815d400766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:54.627162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a1e44-50f8-4b35-b189-8d815d400766 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:54.677649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2799b5-f071-4c88-a6eb-bacf33f35db8 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:54.681415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f478217-06b5-4650-82c4-53c258d6a88c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:54.682045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f478217-06b5-4650-82c4-53c258d6a88c service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:54.682383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f478217-06b5-4650-82c4-53c258d6a88c service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:54.682606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f478217-06b5-4650-82c4-53c258d6a88c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:54.691731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f478217-06b5-4650-82c4-53c258d6a88c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:54.700439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f478217-06b5-4650-82c4-53c258d6a88c service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:54.703125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 917/1843] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:29:54.790133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 927/1844] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:54 2022] POST /identity/v3/projects => generated 397 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:29:54.791599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f478217-06b5-4650-82c4-53c258d6a88c service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:54.801597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f478217-06b5-4650-82c4-53c258d6a88c service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:54.861724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130db75-12fd-48f6-b9eb-74eb0d724cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:54.863749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130db75-12fd-48f6-b9eb-74eb0d724cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:54.863749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130db75-12fd-48f6-b9eb-74eb0d724cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:54.863749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130db75-12fd-48f6-b9eb-74eb0d724cc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:54.863749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130db75-12fd-48f6-b9eb-74eb0d724cc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:54.865343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130db75-12fd-48f6-b9eb-74eb0d724cc6 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:54.959090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 918/1845] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:54 2022] POST /identity/v3/users => generated 305 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:54.963355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a1e44-50f8-4b35-b189-8d815d400766 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:54.975250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a1e44-50f8-4b35-b189-8d815d400766 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:54.980812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b761f4-08ec-4ece-9aef-c425d52fa22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:54.981435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b761f4-08ec-4ece-9aef-c425d52fa22a admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:54.981801 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b761f4-08ec-4ece-9aef-c425d52fa22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:54.981988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b761f4-08ec-4ece-9aef-c425d52fa22a admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:54.982640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b761f4-08ec-4ece-9aef-c425d52fa22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:54.984286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b761f4-08ec-4ece-9aef-c425d52fa22a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:54.994656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 928/1846] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:29:54.999486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130db75-12fd-48f6-b9eb-74eb0d724cc6 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:55.010276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130db75-12fd-48f6-b9eb-74eb0d724cc6 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:55.015242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39687e10-0574-4cd9-aa9b-2dd2f6ae1a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:55.015973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39687e10-0574-4cd9-aa9b-2dd2f6ae1a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:55.016254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39687e10-0574-4cd9-aa9b-2dd2f6ae1a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:55.016676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39687e10-0574-4cd9-aa9b-2dd2f6ae1a30 admin admin] PATH_INFO: `/v3/projects/ed5cf5f74e394b4aa0a1a4d3daff7a12/users/218f71684c8c46268e7bdaa0f5cf8954/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:55.032986 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39687e10-0574-4cd9-aa9b-2dd2f6ae1a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed5cf5f74e394b4aa0a1a4d3daff7a12, user_id=218f71684c8c46268e7bdaa0f5cf8954, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:55.037859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39687e10-0574-4cd9-aa9b-2dd2f6ae1a30 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:55.164358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 919/1847] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:54 2022] PUT /identity/v3/projects/ed5cf5f74e394b4aa0a1a4d3daff7a12/users/218f71684c8c46268e7bdaa0f5cf8954/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:55.169455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b761f4-08ec-4ece-9aef-c425d52fa22a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:55.179177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b761f4-08ec-4ece-9aef-c425d52fa22a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:55.197346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:55.206460 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39687e10-0574-4cd9-aa9b-2dd2f6ae1a30 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:55.209016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-39687e10-0574-4cd9-aa9b-2dd2f6ae1a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:55.259884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14be19d-d3c2-4e55-bd1f-99f44912bf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:55.260500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14be19d-d3c2-4e55-bd1f-99f44912bf77 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:55.260778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14be19d-d3c2-4e55-bd1f-99f44912bf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:55.261049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14be19d-d3c2-4e55-bd1f-99f44912bf77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:55.261727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14be19d-d3c2-4e55-bd1f-99f44912bf77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:55.263333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14be19d-d3c2-4e55-bd1f-99f44912bf77 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:55.274532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 929/1848] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:55.279027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14be19d-d3c2-4e55-bd1f-99f44912bf77 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:55.290042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14be19d-d3c2-4e55-bd1f-99f44912bf77 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:55.295647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667792ef-8ed1-479e-97cc-da766faa0684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:55.296426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667792ef-8ed1-479e-97cc-da766faa0684 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:55.296762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667792ef-8ed1-479e-97cc-da766faa0684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:55.297086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667792ef-8ed1-479e-97cc-da766faa0684 admin admin] PATH_INFO: `/v3/projects/ed5cf5f74e394b4aa0a1a4d3daff7a12/users/218f71684c8c46268e7bdaa0f5cf8954/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:55.299234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667792ef-8ed1-479e-97cc-da766faa0684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed5cf5f74e394b4aa0a1a4d3daff7a12, user_id=218f71684c8c46268e7bdaa0f5cf8954, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:55.301652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39687e10-0574-4cd9-aa9b-2dd2f6ae1a30 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:55.307002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667792ef-8ed1-479e-97cc-da766faa0684 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:55.307107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:55.308182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:55.311345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:55.311698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] PATH_INFO: `/v3/projects/41cdf50f1b0a4256b101c94d735a8998` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:55.313128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41cdf50f1b0a4256b101c94d735a8998)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:55.315850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:55.316368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 930/1849] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:55 2022] PUT /identity/v3/projects/ed5cf5f74e394b4aa0a1a4d3daff7a12/users/218f71684c8c46268e7bdaa0f5cf8954/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:29:55.329446 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb9720-5baa-4a5b-a81f-c9db80a3344b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:55.341612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb9720-5baa-4a5b-a81f-c9db80a3344b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:55.341612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb9720-5baa-4a5b-a81f-c9db80a3344b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:55.369520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-95cb9720-5baa-4a5b-a81f-c9db80a3344b None None] MFA Rules not processed for user `218f71684c8c46268e7bdaa0f5cf8954`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:55.370745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-95cb9720-5baa-4a5b-a81f-c9db80a3344b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:55.466532 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41cdf50f1b0a4256b101c94d735a8998', 'request_id': 'req-fe11409b-7cb8-470d-929b-4eadb2f2d739'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:55.472912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 931/1850] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:55 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:55.473338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41cdf50f1b0a4256b101c94d735a8998', 'request_id': 'req-fe11409b-7cb8-470d-929b-4eadb2f2d739'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:55.480761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb9720-5baa-4a5b-a81f-c9db80a3344b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:55.482395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-95cb9720-5baa-4a5b-a81f-c9db80a3344b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:55.555047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb9720-5baa-4a5b-a81f-c9db80a3344b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:55.579984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41cdf50f1b0a4256b101c94d735a8998', 'request_id': 'req-fe11409b-7cb8-470d-929b-4eadb2f2d739'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:55.594533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] The token cache is being invalidate because project 41cdf50f1b0a4256b101c94d735a8998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:55.594982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:55.596180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:55.604390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 920/1851] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:55 2022] DELETE /identity/v3/projects/41cdf50f1b0a4256b101c94d735a8998 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:29:55.608348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:55.611183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:55.654053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e6768-db74-41e7-bbdf-749eb3ee1efa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:55.655255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e6768-db74-41e7-bbdf-749eb3ee1efa service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:55.658726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e6768-db74-41e7-bbdf-749eb3ee1efa service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:55.659373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e6768-db74-41e7-bbdf-749eb3ee1efa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:55.662414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9a4e6768-db74-41e7-bbdf-749eb3ee1efa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:55.743406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11409b-7cb8-470d-929b-4eadb2f2d739 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:55.765843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:55.765843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:55.765843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:55.765843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] PATH_INFO: `/v3/users/aef0d315d3454710b98cca15f189efc4` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:55.767250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef0d315d3454710b98cca15f189efc4)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:55.769754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:55.778815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e6768-db74-41e7-bbdf-749eb3ee1efa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:55.780456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e6768-db74-41e7-bbdf-749eb3ee1efa service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:55.792114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 932/1852] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:55 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:29:56.060861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef0d315d3454710b98cca15f189efc4'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:56.061619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef0d315d3454710b98cca15f189efc4'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:56.131168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e6768-db74-41e7-bbdf-749eb3ee1efa service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:56.132715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9a4e6768-db74-41e7-bbdf-749eb3ee1efa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:56.143313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e6768-db74-41e7-bbdf-749eb3ee1efa service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:56.147018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69a185-f357-4153-8a13-8c436e051b09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:56.147643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69a185-f357-4153-8a13-8c436e051b09 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:56.147922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69a185-f357-4153-8a13-8c436e051b09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:56.148190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69a185-f357-4153-8a13-8c436e051b09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:56.149906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ea69a185-f357-4153-8a13-8c436e051b09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:56.179877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef0d315d3454710b98cca15f189efc4'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:56.185575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:56.212987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef0d315d3454710b98cca15f189efc4'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:56.214300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef0d315d3454710b98cca15f189efc4'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:56.234689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69a185-f357-4153-8a13-8c436e051b09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:56.236171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69a185-f357-4153-8a13-8c436e051b09 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:56.239380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 921/1853] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:29:55 2022] DELETE /identity/v3/users/aef0d315d3454710b98cca15f189efc4 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:29:56.267958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 933/1854] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:29:56 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:29:56.793065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:56.794927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:56.882170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b306e-e849-45ef-bb56-bbff375d5a59 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:56.887511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:56.888236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:56.888538 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:56.888811 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] PATH_INFO: `/v3/projects/ccf7ef480a5c41988752a408ec363341` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:56.890018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf7ef480a5c41988752a408ec363341)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:56.892434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:57.089817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf7ef480a5c41988752a408ec363341', 'request_id': 'req-28fd70c7-84c6-430d-81a2-a51e8fafd428'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:57.096297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf7ef480a5c41988752a408ec363341', 'request_id': 'req-28fd70c7-84c6-430d-81a2-a51e8fafd428'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:57.242593 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf7ef480a5c41988752a408ec363341', 'request_id': 'req-28fd70c7-84c6-430d-81a2-a51e8fafd428'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:57.266165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] The token cache is being invalidate because project ccf7ef480a5c41988752a408ec363341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:29:57.266671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:57.267723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-28fd70c7-84c6-430d-81a2-a51e8fafd428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:29:57.271039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 922/1855] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:29:56 2022] DELETE /identity/v3/projects/ccf7ef480a5c41988752a408ec363341 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:57.278420 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a039b68-98ea-4389-be72-e95201050daa None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:57.279315 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a039b68-98ea-4389-be72-e95201050daa None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:57.279315 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a039b68-98ea-4389-be72-e95201050daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:57.304596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4a039b68-98ea-4389-be72-e95201050daa None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:57.305869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4a039b68-98ea-4389-be72-e95201050daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:57.394620 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 934/1856] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:29:57.406676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab554da-fb0d-4213-b8d7-69eb0589a341 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:57.406955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab554da-fb0d-4213-b8d7-69eb0589a341 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:57.407275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab554da-fb0d-4213-b8d7-69eb0589a341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:57.429961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-3ab554da-fb0d-4213-b8d7-69eb0589a341 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:57.431166 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3ab554da-fb0d-4213-b8d7-69eb0589a341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:57.439310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 923/1857] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:29:57.443736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a039b68-98ea-4389-be72-e95201050daa None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:57.453498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a039b68-98ea-4389-be72-e95201050daa None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:57.458258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1aa8d-967b-4bae-b3e2-44193f8bdaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:57.458859 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1aa8d-967b-4bae-b3e2-44193f8bdaa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:57.459133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1aa8d-967b-4bae-b3e2-44193f8bdaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:57.459400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1aa8d-967b-4bae-b3e2-44193f8bdaa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:57.460081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1aa8d-967b-4bae-b3e2-44193f8bdaa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:29:57.460378 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1aa8d-967b-4bae-b3e2-44193f8bdaa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:57.462025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1aa8d-967b-4bae-b3e2-44193f8bdaa3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:57.475025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 935/1858] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:29:57.479807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab554da-fb0d-4213-b8d7-69eb0589a341 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:57.490734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab554da-fb0d-4213-b8d7-69eb0589a341 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:57.496180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be881c4-9ffa-4c06-8e24-59492f2d52c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:57.496987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be881c4-9ffa-4c06-8e24-59492f2d52c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:57.497341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be881c4-9ffa-4c06-8e24-59492f2d52c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:57.497667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be881c4-9ffa-4c06-8e24-59492f2d52c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:57.498480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be881c4-9ffa-4c06-8e24-59492f2d52c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:57.500649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be881c4-9ffa-4c06-8e24-59492f2d52c6 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:57.657633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 924/1859] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:57 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:57.659775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1aa8d-967b-4bae-b3e2-44193f8bdaa3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:57.672707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1aa8d-967b-4bae-b3e2-44193f8bdaa3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:57.761368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea45ad-1a67-459e-b1c8-92416e0429f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:57.762608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea45ad-1a67-459e-b1c8-92416e0429f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:57.763105 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea45ad-1a67-459e-b1c8-92416e0429f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:57.763734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea45ad-1a67-459e-b1c8-92416e0429f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:57.764869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea45ad-1a67-459e-b1c8-92416e0429f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:57.768692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea45ad-1a67-459e-b1c8-92416e0429f8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:57.808862 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:29:57.932303 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 936/1860] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:57 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:57.933532 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be881c4-9ffa-4c06-8e24-59492f2d52c6 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:57.946977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be881c4-9ffa-4c06-8e24-59492f2d52c6 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:57.954723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad622171-1d31-4660-a2fb-a9d5375b654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:57.955478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad622171-1d31-4660-a2fb-a9d5375b654b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:57.956035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad622171-1d31-4660-a2fb-a9d5375b654b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:57.956498 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad622171-1d31-4660-a2fb-a9d5375b654b admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:57.957310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad622171-1d31-4660-a2fb-a9d5375b654b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:57.959718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad622171-1d31-4660-a2fb-a9d5375b654b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:57.971889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 925/1861] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:29:57.976775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea45ad-1a67-459e-b1c8-92416e0429f8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:57.987278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea45ad-1a67-459e-b1c8-92416e0429f8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:57.992091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594331-e03f-4d57-a9d1-c0243c61e4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:57.992764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594331-e03f-4d57-a9d1-c0243c61e4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:57.993114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594331-e03f-4d57-a9d1-c0243c61e4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:57.993441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594331-e03f-4d57-a9d1-c0243c61e4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:57.994145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594331-e03f-4d57-a9d1-c0243c61e4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:57.995989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594331-e03f-4d57-a9d1-c0243c61e4c3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:58.010941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 937/1862] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:58.012274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad622171-1d31-4660-a2fb-a9d5375b654b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:58.023148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad622171-1d31-4660-a2fb-a9d5375b654b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:58.028407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f7d5f-85f1-4d7a-bc01-842c66fec474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:58.029083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f7d5f-85f1-4d7a-bc01-842c66fec474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:58.029579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f7d5f-85f1-4d7a-bc01-842c66fec474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:58.029788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f7d5f-85f1-4d7a-bc01-842c66fec474 admin admin] PATH_INFO: `/v3/projects/bade450ba43440f58c22858d8e1af7e1/users/4b7c0078ecc441e6b6a1392ab8a1c528/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:58.046646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f7d5f-85f1-4d7a-bc01-842c66fec474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bade450ba43440f58c22858d8e1af7e1, user_id=4b7c0078ecc441e6b6a1392ab8a1c528, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:58.049620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f7d5f-85f1-4d7a-bc01-842c66fec474 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:58.180587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 926/1863] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:58 2022] PUT /identity/v3/projects/bade450ba43440f58c22858d8e1af7e1/users/4b7c0078ecc441e6b6a1392ab8a1c528/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:58.191464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9123998-8cae-4cd6-be45-a14f2b9e6661 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:58.191657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9123998-8cae-4cd6-be45-a14f2b9e6661 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:58.192537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9123998-8cae-4cd6-be45-a14f2b9e6661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:58.241427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f9123998-8cae-4cd6-be45-a14f2b9e6661 None None] MFA Rules not processed for user `4b7c0078ecc441e6b6a1392ab8a1c528`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:58.241427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f9123998-8cae-4cd6-be45-a14f2b9e6661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:29:58.326358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 938/1864] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:29:58.331277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f7d5f-85f1-4d7a-bc01-842c66fec474 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:58.422034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f7d5f-85f1-4d7a-bc01-842c66fec474 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:58.485805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b82501-7878-4d55-ab00-ad07053e4249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:58.485805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b82501-7878-4d55-ab00-ad07053e4249 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:58.485805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b82501-7878-4d55-ab00-ad07053e4249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:58.485805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b82501-7878-4d55-ab00-ad07053e4249 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:58.487149 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b82501-7878-4d55-ab00-ad07053e4249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:58.487149 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b82501-7878-4d55-ab00-ad07053e4249 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:59.401574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 927/1865] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:29:58 2022] POST /identity/v3/projects => generated 379 bytes in 1068 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:29:59.403273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9123998-8cae-4cd6-be45-a14f2b9e6661 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:59.416855 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9123998-8cae-4cd6-be45-a14f2b9e6661 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:59.500069 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fb838-1f75-45b9-bcea-2c7a4deaca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:59.500687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fb838-1f75-45b9-bcea-2c7a4deaca2d admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:59.501011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fb838-1f75-45b9-bcea-2c7a4deaca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:59.501290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fb838-1f75-45b9-bcea-2c7a4deaca2d admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:59.501921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628fb838-1f75-45b9-bcea-2c7a4deaca2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:59.503903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628fb838-1f75-45b9-bcea-2c7a4deaca2d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:59.664328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 939/1866] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:29:59 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:29:59.669373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b82501-7878-4d55-ab00-ad07053e4249 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:59.680408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b82501-7878-4d55-ab00-ad07053e4249 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:59.689564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e66d3-6053-4154-b667-78671f09f834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:59.690406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e66d3-6053-4154-b667-78671f09f834 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:59.692526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e66d3-6053-4154-b667-78671f09f834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:59.692526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e66d3-6053-4154-b667-78671f09f834 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:59.693081 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e66d3-6053-4154-b667-78671f09f834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:59.695119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e66d3-6053-4154-b667-78671f09f834 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:59.722698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 928/1867] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:29:59.729634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fb838-1f75-45b9-bcea-2c7a4deaca2d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:59.739805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fb838-1f75-45b9-bcea-2c7a4deaca2d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:59.744967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1593f-6568-4858-95d3-fcddfd8a72fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:59.745572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1593f-6568-4858-95d3-fcddfd8a72fa admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:59.746018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1593f-6568-4858-95d3-fcddfd8a72fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:59.746294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1593f-6568-4858-95d3-fcddfd8a72fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:59.746935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1593f-6568-4858-95d3-fcddfd8a72fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:59.748567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1593f-6568-4858-95d3-fcddfd8a72fa admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:59.766491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 940/1868] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:29:59.767979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e66d3-6053-4154-b667-78671f09f834 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:29:59.784313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e66d3-6053-4154-b667-78671f09f834 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:29:59.791168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d155789-3398-40d2-a71b-473702882d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:29:59.795493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d155789-3398-40d2-a71b-473702882d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:59.795493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d155789-3398-40d2-a71b-473702882d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:59.795493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d155789-3398-40d2-a71b-473702882d3c admin admin] PATH_INFO: `/v3/projects/e5372c3103f544ccaccb6ab3be83f16c/users/4f143567da3946c5a518e3197b4a303d/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:59.810692 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d155789-3398-40d2-a71b-473702882d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5372c3103f544ccaccb6ab3be83f16c, user_id=4f143567da3946c5a518e3197b4a303d, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:29:59.813640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d155789-3398-40d2-a71b-473702882d3c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:29:59.901439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 929/1869] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:29:59 2022] PUT /identity/v3/projects/e5372c3103f544ccaccb6ab3be83f16c/users/4f143567da3946c5a518e3197b4a303d/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:29:59.913652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44770d9-cf18-41d4-8f53-ce4210c58d6a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:29:59.913939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44770d9-cf18-41d4-8f53-ce4210c58d6a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:29:59.914219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44770d9-cf18-41d4-8f53-ce4210c58d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:29:59.943225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e44770d9-cf18-41d4-8f53-ce4210c58d6a None None] MFA Rules not processed for user `4f143567da3946c5a518e3197b4a303d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:29:59.944479 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e44770d9-cf18-41d4-8f53-ce4210c58d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:00.010788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 941/1870] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:29:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:00.016012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d155789-3398-40d2-a71b-473702882d3c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:00.023716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:00.095025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d155789-3398-40d2-a71b-473702882d3c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:00.100205 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004043f-32c6-4895-8cd1-b43463ab3ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:00.100822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004043f-32c6-4895-8cd1-b43463ab3ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:00.102893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004043f-32c6-4895-8cd1-b43463ab3ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:00.102893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004043f-32c6-4895-8cd1-b43463ab3ab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:00.102893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004043f-32c6-4895-8cd1-b43463ab3ab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:00.104578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004043f-32c6-4895-8cd1-b43463ab3ab7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:00.291825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 930/1871] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:00 2022] POST /identity/v3/projects => generated 377 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:00.297901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44770d9-cf18-41d4-8f53-ce4210c58d6a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:00.318300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44770d9-cf18-41d4-8f53-ce4210c58d6a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:00.417347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4411ec-9de7-4d47-a6a8-94a0e4531c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:00.418115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4411ec-9de7-4d47-a6a8-94a0e4531c7e admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:00.418468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4411ec-9de7-4d47-a6a8-94a0e4531c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:00.418810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4411ec-9de7-4d47-a6a8-94a0e4531c7e admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:00.419642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4411ec-9de7-4d47-a6a8-94a0e4531c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:00.421833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4411ec-9de7-4d47-a6a8-94a0e4531c7e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:00.550696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 942/1872] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:00 2022] POST /identity/v3/users => generated 295 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:00.555569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004043f-32c6-4895-8cd1-b43463ab3ab7 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:00.577589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004043f-32c6-4895-8cd1-b43463ab3ab7 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:00.577589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8bf82-4b00-4400-810f-22ac16689382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:00.578022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8bf82-4b00-4400-810f-22ac16689382 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:00.578162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8bf82-4b00-4400-810f-22ac16689382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:00.578476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8bf82-4b00-4400-810f-22ac16689382 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:00.579192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8bf82-4b00-4400-810f-22ac16689382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:00.581360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8bf82-4b00-4400-810f-22ac16689382 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:00.592143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 931/1873] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:00.597666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4411ec-9de7-4d47-a6a8-94a0e4531c7e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:00.611251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4411ec-9de7-4d47-a6a8-94a0e4531c7e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:00.617678 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd63c5-e3f7-4681-997e-5ca7956849d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:00.618891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd63c5-e3f7-4681-997e-5ca7956849d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:00.619334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd63c5-e3f7-4681-997e-5ca7956849d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:00.619779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd63c5-e3f7-4681-997e-5ca7956849d7 admin admin] PATH_INFO: `/v3/projects/db29621521a14d16b05afc7ceb841d8e/users/0284c828e29f41aa86bb4c20e5bc0392/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:00.649188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd63c5-e3f7-4681-997e-5ca7956849d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db29621521a14d16b05afc7ceb841d8e, user_id=0284c828e29f41aa86bb4c20e5bc0392, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:00.652183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd63c5-e3f7-4681-997e-5ca7956849d7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:00.776514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 943/1874] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:00 2022] PUT /identity/v3/projects/db29621521a14d16b05afc7ceb841d8e/users/0284c828e29f41aa86bb4c20e5bc0392/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:00.785446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8bf82-4b00-4400-810f-22ac16689382 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:00.826491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8bf82-4b00-4400-810f-22ac16689382 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:00.979406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978aeda-2c0d-4fa2-9669-ff8e1e2d389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:00.980252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978aeda-2c0d-4fa2-9669-ff8e1e2d389e admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:00.980633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978aeda-2c0d-4fa2-9669-ff8e1e2d389e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:00.981077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978aeda-2c0d-4fa2-9669-ff8e1e2d389e admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:00.981863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978aeda-2c0d-4fa2-9669-ff8e1e2d389e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:00.984118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978aeda-2c0d-4fa2-9669-ff8e1e2d389e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:01.006550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 932/1875] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:01.010518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd63c5-e3f7-4681-997e-5ca7956849d7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:01.160452 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd63c5-e3f7-4681-997e-5ca7956849d7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:01.160452 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3549eb-2c88-40c4-bfd4-3f1825844e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:01.160452 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3549eb-2c88-40c4-bfd4-3f1825844e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:01.160452 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3549eb-2c88-40c4-bfd4-3f1825844e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:01.164093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3549eb-2c88-40c4-bfd4-3f1825844e81 admin admin] PATH_INFO: `/v3/projects/db29621521a14d16b05afc7ceb841d8e/users/0284c828e29f41aa86bb4c20e5bc0392/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:01.164093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3549eb-2c88-40c4-bfd4-3f1825844e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db29621521a14d16b05afc7ceb841d8e, user_id=0284c828e29f41aa86bb4c20e5bc0392, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:01.164093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3549eb-2c88-40c4-bfd4-3f1825844e81 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:01.164093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 944/1876] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:01 2022] PUT /identity/v3/projects/db29621521a14d16b05afc7ceb841d8e/users/0284c828e29f41aa86bb4c20e5bc0392/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:01.164663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa9e18-a4ce-4d67-b54f-b4961b1c5eb1 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:01.164663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa9e18-a4ce-4d67-b54f-b4961b1c5eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:01.164663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa9e18-a4ce-4d67-b54f-b4961b1c5eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:01.181520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-17aa9e18-a4ce-4d67-b54f-b4961b1c5eb1 None None] MFA Rules not processed for user `0284c828e29f41aa86bb4c20e5bc0392`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:01.183133 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-17aa9e18-a4ce-4d67-b54f-b4961b1c5eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:01.263968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 933/1877] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:01.323465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3549eb-2c88-40c4-bfd4-3f1825844e81 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:01.325737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4c3549eb-2c88-40c4-bfd4-3f1825844e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:01.401308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3549eb-2c88-40c4-bfd4-3f1825844e81 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:01.405784 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454de8e1-bb3c-45f4-8cd1-df1c63ba49e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:01.407504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454de8e1-bb3c-45f4-8cd1-df1c63ba49e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:01.407822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454de8e1-bb3c-45f4-8cd1-df1c63ba49e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:01.408160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454de8e1-bb3c-45f4-8cd1-df1c63ba49e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:01.429614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa9e18-a4ce-4d67-b54f-b4961b1c5eb1 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:01.431440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454de8e1-bb3c-45f4-8cd1-df1c63ba49e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:01.432181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-17aa9e18-a4ce-4d67-b54f-b4961b1c5eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:01.433453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454de8e1-bb3c-45f4-8cd1-df1c63ba49e5 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:01.447799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 945/1878] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:01.540617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa9e18-a4ce-4d67-b54f-b4961b1c5eb1 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:01.545373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:01.546037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:01.546327 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:01.546597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] PATH_INFO: `/v3/users/71f94e660c1d4e5ab35f8a1b7d78a84e` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:01.547898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f94e660c1d4e5ab35f8a1b7d78a84e)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:01.549907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:01.589629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:01.609625 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454de8e1-bb3c-45f4-8cd1-df1c63ba49e5 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:01.612261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-454de8e1-bb3c-45f4-8cd1-df1c63ba49e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:01.627764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454de8e1-bb3c-45f4-8cd1-df1c63ba49e5 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:01.633365 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6154e-5ac3-4b29-b764-1070dfd3c266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:01.634065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6154e-5ac3-4b29-b764-1070dfd3c266 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:01.634413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6154e-5ac3-4b29-b764-1070dfd3c266 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:01.634777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6154e-5ac3-4b29-b764-1070dfd3c266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:01.707705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6154e-5ac3-4b29-b764-1070dfd3c266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:01.709570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6154e-5ac3-4b29-b764-1070dfd3c266 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:01.720008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 946/1879] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:01.869758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f94e660c1d4e5ab35f8a1b7d78a84e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:01.870687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f94e660c1d4e5ab35f8a1b7d78a84e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:01.985722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f94e660c1d4e5ab35f8a1b7d78a84e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:02.000688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f94e660c1d4e5ab35f8a1b7d78a84e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:02.001420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f94e660c1d4e5ab35f8a1b7d78a84e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:02.016737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 934/1880] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:01 2022] DELETE /identity/v3/users/71f94e660c1d4e5ab35f8a1b7d78a84e => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:02.024062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efc44f-1e28-4d74-a3a4-57b6231ddb41 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:02.025355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efc44f-1e28-4d74-a3a4-57b6231ddb41 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:02.026146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efc44f-1e28-4d74-a3a4-57b6231ddb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:02.080836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b8efc44f-1e28-4d74-a3a4-57b6231ddb41 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:02.082263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b8efc44f-1e28-4d74-a3a4-57b6231ddb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:02.155643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 947/1881] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:02.169183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:02.174185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:02.269035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcfeb5-54fd-463c-aa38-36f52211dd8e admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:02.277394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c17552-755e-4435-b97e-b303722892ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:02.282156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c17552-755e-4435-b97e-b303722892ce service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:02.282156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c17552-755e-4435-b97e-b303722892ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:02.282156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c17552-755e-4435-b97e-b303722892ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:02.297305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c17552-755e-4435-b97e-b303722892ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:02.300243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c17552-755e-4435-b97e-b303722892ce service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:02.320037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 935/1882] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:02.979198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efc44f-1e28-4d74-a3a4-57b6231ddb41 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:02.980963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b8efc44f-1e28-4d74-a3a4-57b6231ddb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:02.994661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:03.001045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efc44f-1e28-4d74-a3a4-57b6231ddb41 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:03.010116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:03.010116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:03.010116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:03.010116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] PATH_INFO: `/v3/users/db49e7b50f304c94be3a4ad96598bcf2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:03.010879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db49e7b50f304c94be3a4ad96598bcf2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:03.013594 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:03.184243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c17552-755e-4435-b97e-b303722892ce service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:03.185959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-78c17552-755e-4435-b97e-b303722892ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:03.201928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c17552-755e-4435-b97e-b303722892ce service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:03.207869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:03.208926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:03.209418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:03.210321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] PATH_INFO: `/v3/projects/3504eb93c0ce4e40a219359a85f230fc` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:03.212294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3504eb93c0ce4e40a219359a85f230fc)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:03.214502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:03.330510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db49e7b50f304c94be3a4ad96598bcf2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.331374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db49e7b50f304c94be3a4ad96598bcf2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.368340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3504eb93c0ce4e40a219359a85f230fc', 'request_id': 'req-9160d7ad-72f2-419c-9b57-15ff365b7efb'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.374647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3504eb93c0ce4e40a219359a85f230fc', 'request_id': 'req-9160d7ad-72f2-419c-9b57-15ff365b7efb'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.423337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db49e7b50f304c94be3a4ad96598bcf2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.441733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db49e7b50f304c94be3a4ad96598bcf2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.454329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2b04bca3-12e3-4e4a-8347-7583c262ad06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db49e7b50f304c94be3a4ad96598bcf2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.490986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 948/1883] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:02 2022] DELETE /identity/v3/users/db49e7b50f304c94be3a4ad96598bcf2 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:03.493150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b6fa8-34d4-49f9-a384-a79fb6b76d96 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:03.493651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b6fa8-34d4-49f9-a384-a79fb6b76d96 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:03.494096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b6fa8-34d4-49f9-a384-a79fb6b76d96 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:03.511419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 949/1884] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:30:03.511419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d048dd-58cc-4bf8-b1c7-0fa853d72fa3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:03.511419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d048dd-58cc-4bf8-b1c7-0fa853d72fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:03.511419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d048dd-58cc-4bf8-b1c7-0fa853d72fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:03.522592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-72d048dd-58cc-4bf8-b1c7-0fa853d72fa3 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:03.524135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-72d048dd-58cc-4bf8-b1c7-0fa853d72fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:03.533875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3504eb93c0ce4e40a219359a85f230fc', 'request_id': 'req-9160d7ad-72f2-419c-9b57-15ff365b7efb'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.564642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] The token cache is being invalidate because project 3504eb93c0ce4e40a219359a85f230fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:03.569477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.569477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9160d7ad-72f2-419c-9b57-15ff365b7efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.569477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 936/1885] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:03 2022] DELETE /identity/v3/projects/3504eb93c0ce4e40a219359a85f230fc => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:03.570584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756a96b-9509-4f1a-b323-78f623a10d8a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:03.571091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756a96b-9509-4f1a-b323-78f623a10d8a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:03.571643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756a96b-9509-4f1a-b323-78f623a10d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:03.601303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-4756a96b-9509-4f1a-b323-78f623a10d8a None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:03.603911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4756a96b-9509-4f1a-b323-78f623a10d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:03.623602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 950/1886] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:03.623602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d048dd-58cc-4bf8-b1c7-0fa853d72fa3 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:03.623602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-72d048dd-58cc-4bf8-b1c7-0fa853d72fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:03.640223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d048dd-58cc-4bf8-b1c7-0fa853d72fa3 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:03.645297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:03.646095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:03.646566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:03.646942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] PATH_INFO: `/v3/users/dd1d45a23e4943a5b537d90647983563` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:03.648272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd1d45a23e4943a5b537d90647983563)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:03.650573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:03.704585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 937/1887] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:30:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:03.705358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756a96b-9509-4f1a-b323-78f623a10d8a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:03.706636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4756a96b-9509-4f1a-b323-78f623a10d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:03.766865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756a96b-9509-4f1a-b323-78f623a10d8a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:03.771704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37175c1b-e0dc-41b3-9e00-22f23830524d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:03.772399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37175c1b-e0dc-41b3-9e00-22f23830524d service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:03.772679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37175c1b-e0dc-41b3-9e00-22f23830524d service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:03.772973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37175c1b-e0dc-41b3-9e00-22f23830524d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:03.782424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37175c1b-e0dc-41b3-9e00-22f23830524d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:03.783976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37175c1b-e0dc-41b3-9e00-22f23830524d service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:03.795008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 938/1888] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:03.798389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37175c1b-e0dc-41b3-9e00-22f23830524d service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:03.798389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-37175c1b-e0dc-41b3-9e00-22f23830524d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:03.813529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37175c1b-e0dc-41b3-9e00-22f23830524d service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:03.821004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674084b2-d524-41cc-9404-a165189c412a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:03.821630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674084b2-d524-41cc-9404-a165189c412a service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:03.821906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674084b2-d524-41cc-9404-a165189c412a service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:03.822178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674084b2-d524-41cc-9404-a165189c412a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:03.831481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674084b2-d524-41cc-9404-a165189c412a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:03.838371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674084b2-d524-41cc-9404-a165189c412a service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:03.845544 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 939/1889] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:03.915744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd1d45a23e4943a5b537d90647983563'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:03.918229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd1d45a23e4943a5b537d90647983563'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:04.031916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd1d45a23e4943a5b537d90647983563'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:04.060063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd1d45a23e4943a5b537d90647983563'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:04.061177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd1d45a23e4943a5b537d90647983563'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:04.101000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 951/1890] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:03 2022] DELETE /identity/v3/users/dd1d45a23e4943a5b537d90647983563 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:04.581317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674084b2-d524-41cc-9404-a165189c412a service nova] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:04.583900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-674084b2-d524-41cc-9404-a165189c412a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:04.600587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674084b2-d524-41cc-9404-a165189c412a service nova] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:04.672863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:04.673596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:04.673955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:04.674308 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] PATH_INFO: `/v3/projects/6c414ade18ea4ad2a562ae77802b472f` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:04.675694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c414ade18ea4ad2a562ae77802b472f)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:04.679323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:04.885184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c414ade18ea4ad2a562ae77802b472f', 'request_id': 'req-dbb0e2a6-5f83-4e07-858c-940008453751'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:04.895706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c414ade18ea4ad2a562ae77802b472f', 'request_id': 'req-dbb0e2a6-5f83-4e07-858c-940008453751'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:04.902336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:04.903802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:04.919195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6399d3-291d-4d62-be13-3fe0961e939d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:04.985194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:04.985750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:04.986031 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:04.986310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] PATH_INFO: `/v3/users/2fd7fb2bd0f043ec9444c2c73c700513` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:04.987465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd7fb2bd0f043ec9444c2c73c700513)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:04.990045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:05.019759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c414ade18ea4ad2a562ae77802b472f', 'request_id': 'req-dbb0e2a6-5f83-4e07-858c-940008453751'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.055388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] The token cache is being invalidate because project 6c414ade18ea4ad2a562ae77802b472f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:05.067950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.070940 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.075991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 940/1891] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:04 2022] DELETE /identity/v3/projects/6c414ade18ea4ad2a562ae77802b472f => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:05.076994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:05.088002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:05.102409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0e2a6-5f83-4e07-858c-940008453751 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:05.107525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:05.108219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:05.108688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:05.109629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] PATH_INFO: `/v3/projects/cdbd86ce99b34d63b53ec02ebcbeeeaf` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:05.111171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdbd86ce99b34d63b53ec02ebcbeeeaf)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:05.113572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:05.147293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:05.196284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd7fb2bd0f043ec9444c2c73c700513'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.197187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd7fb2bd0f043ec9444c2c73c700513'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.304665 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdbd86ce99b34d63b53ec02ebcbeeeaf', 'request_id': 'req-069bc0c7-6481-45fb-8535-a451f8887ef6'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.311289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdbd86ce99b34d63b53ec02ebcbeeeaf', 'request_id': 'req-069bc0c7-6481-45fb-8535-a451f8887ef6'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.318304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd7fb2bd0f043ec9444c2c73c700513'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.341007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd7fb2bd0f043ec9444c2c73c700513'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.342123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd7fb2bd0f043ec9444c2c73c700513'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.370239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 952/1892] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:04 2022] DELETE /identity/v3/users/2fd7fb2bd0f043ec9444c2c73c700513 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:05.373676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:05.377851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:05.420076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdbd86ce99b34d63b53ec02ebcbeeeaf', 'request_id': 'req-069bc0c7-6481-45fb-8535-a451f8887ef6'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.439169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] The token cache is being invalidate because project cdbd86ce99b34d63b53ec02ebcbeeeaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:05.439448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.440354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-069bc0c7-6481-45fb-8535-a451f8887ef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.442079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 941/1893] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:05 2022] DELETE /identity/v3/projects/cdbd86ce99b34d63b53ec02ebcbeeeaf => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:05.443526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd3faa-9a67-4c49-a3ad-4f78e06ae0dc None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:05.444017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd3faa-9a67-4c49-a3ad-4f78e06ae0dc None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:05.444442 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd3faa-9a67-4c49-a3ad-4f78e06ae0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:05.474545 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-fedd3faa-9a67-4c49-a3ad-4f78e06ae0dc None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:05.476230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fedd3faa-9a67-4c49-a3ad-4f78e06ae0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:05.476653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10072e-5b1e-4839-859a-d1ed8241d363 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:05.481790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:05.482438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:05.482713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:05.482988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] PATH_INFO: `/v3/users/218f71684c8c46268e7bdaa0f5cf8954` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:05.484177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218f71684c8c46268e7bdaa0f5cf8954)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:05.486147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:05.488653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 942/1894] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:05.492120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e956ab3-6f58-4895-a0b1-77034ac21e1a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:05.492120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e956ab3-6f58-4895-a0b1-77034ac21e1a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:05.492120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e956ab3-6f58-4895-a0b1-77034ac21e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:05.515315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7e956ab3-6f58-4895-a0b1-77034ac21e1a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:05.516691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7e956ab3-6f58-4895-a0b1-77034ac21e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:05.527996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 943/1895] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:05.528853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e956ab3-6f58-4895-a0b1-77034ac21e1a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:05.530614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7e956ab3-6f58-4895-a0b1-77034ac21e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:05.595264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e956ab3-6f58-4895-a0b1-77034ac21e1a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:05.599074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e60ce-a996-483f-b686-f003c4407eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:05.599733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e60ce-a996-483f-b686-f003c4407eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:05.600266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e60ce-a996-483f-b686-f003c4407eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:05.600541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e60ce-a996-483f-b686-f003c4407eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:05.610369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e60ce-a996-483f-b686-f003c4407eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:05.611834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e60ce-a996-483f-b686-f003c4407eb0 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:05.621310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 944/1896] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:05.622551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451b1e6-e0db-4328-8ce2-7a5d28d5fdcb None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:05.622820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451b1e6-e0db-4328-8ce2-7a5d28d5fdcb None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:05.623115 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451b1e6-e0db-4328-8ce2-7a5d28d5fdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:05.643668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:05.651305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6451b1e6-e0db-4328-8ce2-7a5d28d5fdcb None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:05.654436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6451b1e6-e0db-4328-8ce2-7a5d28d5fdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:05.669540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 945/1897] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:05.670764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451b1e6-e0db-4328-8ce2-7a5d28d5fdcb None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:05.680853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451b1e6-e0db-4328-8ce2-7a5d28d5fdcb None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:05.685884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e910e-8087-46d2-8f7a-95332ca7d9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:05.686818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e910e-8087-46d2-8f7a-95332ca7d9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:05.687215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e910e-8087-46d2-8f7a-95332ca7d9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:05.687600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e910e-8087-46d2-8f7a-95332ca7d9d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:05.688547 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e910e-8087-46d2-8f7a-95332ca7d9d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:05.689140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e910e-8087-46d2-8f7a-95332ca7d9d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:05.690739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e910e-8087-46d2-8f7a-95332ca7d9d3 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:05.704870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 946/1898] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:30:05.710037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e910e-8087-46d2-8f7a-95332ca7d9d3 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:05.723501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e910e-8087-46d2-8f7a-95332ca7d9d3 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:05.737858 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf8b21-64f2-4095-a56a-0a9fe87b3f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:05.738646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf8b21-64f2-4095-a56a-0a9fe87b3f32 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:05.740138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf8b21-64f2-4095-a56a-0a9fe87b3f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:05.742649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf8b21-64f2-4095-a56a-0a9fe87b3f32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:05.742649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf8b21-64f2-4095-a56a-0a9fe87b3f32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:05.744571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf8b21-64f2-4095-a56a-0a9fe87b3f32 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:05.758532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218f71684c8c46268e7bdaa0f5cf8954'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.759709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218f71684c8c46268e7bdaa0f5cf8954'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.826409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218f71684c8c46268e7bdaa0f5cf8954'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.844616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 947/1899] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:05 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:05.850161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf8b21-64f2-4095-a56a-0a9fe87b3f32 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:05.853357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cdcf8b21-64f2-4095-a56a-0a9fe87b3f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:05.857270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218f71684c8c46268e7bdaa0f5cf8954'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.858516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218f71684c8c46268e7bdaa0f5cf8954'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:05.887621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 953/1900] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:05 2022] DELETE /identity/v3/users/218f71684c8c46268e7bdaa0f5cf8954 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:06.045478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf8b21-64f2-4095-a56a-0a9fe87b3f32 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:06.181646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f946e23-a4ad-4705-89e0-e7fa63ca5177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:06.183572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f946e23-a4ad-4705-89e0-e7fa63ca5177 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:06.184286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f946e23-a4ad-4705-89e0-e7fa63ca5177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:06.184706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f946e23-a4ad-4705-89e0-e7fa63ca5177 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:06.185605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f946e23-a4ad-4705-89e0-e7fa63ca5177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:06.200032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f946e23-a4ad-4705-89e0-e7fa63ca5177 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:06.345321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 948/1901] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:05 2022] POST /identity/v3/users => generated 293 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:06.348084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:06.365022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc72c4-fa21-4803-9d51-510783139b05 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:06.374733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d3229-c876-4d5e-808b-227c4277b0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:06.377477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d3229-c876-4d5e-808b-227c4277b0ed admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:06.380428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d3229-c876-4d5e-808b-227c4277b0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:06.380896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d3229-c876-4d5e-808b-227c4277b0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:06.383127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d3229-c876-4d5e-808b-227c4277b0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:06.385279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d3229-c876-4d5e-808b-227c4277b0ed admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:06.398629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 954/1902] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:06.409711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f946e23-a4ad-4705-89e0-e7fa63ca5177 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:06.433606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f946e23-a4ad-4705-89e0-e7fa63ca5177 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:06.443405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8727a5-449f-4b8f-aed7-5ecda504ed1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:06.444750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8727a5-449f-4b8f-aed7-5ecda504ed1b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:06.449946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8727a5-449f-4b8f-aed7-5ecda504ed1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:06.449946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8727a5-449f-4b8f-aed7-5ecda504ed1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:06.449946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8727a5-449f-4b8f-aed7-5ecda504ed1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:06.449946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8727a5-449f-4b8f-aed7-5ecda504ed1b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:06.466454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d3229-c876-4d5e-808b-227c4277b0ed admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:06.477967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 949/1903] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:06.481296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d3229-c876-4d5e-808b-227c4277b0ed admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:06.490823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe41205-c15b-4f56-88ad-efdbbd1f5654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:06.495989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe41205-c15b-4f56-88ad-efdbbd1f5654 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:06.495989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe41205-c15b-4f56-88ad-efdbbd1f5654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:06.495989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe41205-c15b-4f56-88ad-efdbbd1f5654 admin admin] PATH_INFO: `/v3/projects/38c92e2b51b2477fb181f731e7c412c7/users/2d6bad4ac24f41db8b30dc42cf8b97f9/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:06.517714 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe41205-c15b-4f56-88ad-efdbbd1f5654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c92e2b51b2477fb181f731e7c412c7, user_id=2d6bad4ac24f41db8b30dc42cf8b97f9, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:06.521494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe41205-c15b-4f56-88ad-efdbbd1f5654 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:06.688663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 955/1904] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:06 2022] PUT /identity/v3/projects/38c92e2b51b2477fb181f731e7c412c7/users/2d6bad4ac24f41db8b30dc42cf8b97f9/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:06.700597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc301b-64bf-480f-a2ae-2045a19efd41 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:06.701412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc301b-64bf-480f-a2ae-2045a19efd41 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:06.701412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc301b-64bf-480f-a2ae-2045a19efd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:06.778145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-43bc301b-64bf-480f-a2ae-2045a19efd41 None None] MFA Rules not processed for user `2d6bad4ac24f41db8b30dc42cf8b97f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:06.780826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-43bc301b-64bf-480f-a2ae-2045a19efd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:06.908107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe41205-c15b-4f56-88ad-efdbbd1f5654 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:06.910012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-afe41205-c15b-4f56-88ad-efdbbd1f5654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:06.971503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 950/1905] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:06.972534 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc301b-64bf-480f-a2ae-2045a19efd41 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:07.131465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc301b-64bf-480f-a2ae-2045a19efd41 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:07.136709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe41205-c15b-4f56-88ad-efdbbd1f5654 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:07.144007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2da41-66b1-4f1d-89d5-7597263da7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:07.144675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2da41-66b1-4f1d-89d5-7597263da7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:07.144956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2da41-66b1-4f1d-89d5-7597263da7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:07.145231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2da41-66b1-4f1d-89d5-7597263da7c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:07.145886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2da41-66b1-4f1d-89d5-7597263da7c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:07.146940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:07.147747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:07.147915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2da41-66b1-4f1d-89d5-7597263da7c5 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:07.148144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:07.148491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] PATH_INFO: `/v3/projects/ba12976666de475194a11697e4183b97` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:07.149800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba12976666de475194a11697e4183b97)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:07.151651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:07.291392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 951/1906] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:06 2022] POST /identity/v3/projects => generated 385 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:07.292354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2da41-66b1-4f1d-89d5-7597263da7c5 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:07.293943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-dbc2da41-66b1-4f1d-89d5-7597263da7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:07.303159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba12976666de475194a11697e4183b97', 'request_id': 'req-66f4e0fc-f083-4299-80a0-7a11333e92d9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.311103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba12976666de475194a11697e4183b97', 'request_id': 'req-66f4e0fc-f083-4299-80a0-7a11333e92d9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.378460 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2da41-66b1-4f1d-89d5-7597263da7c5 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:07.383750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:07.384395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:07.384702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:07.384977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] PATH_INFO: `/v3/users/4b7c0078ecc441e6b6a1392ab8a1c528` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:07.386200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b7c0078ecc441e6b6a1392ab8a1c528)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:07.388203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:07.403041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba12976666de475194a11697e4183b97', 'request_id': 'req-66f4e0fc-f083-4299-80a0-7a11333e92d9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.416091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] The token cache is being invalidate because project ba12976666de475194a11697e4183b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:07.416383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.417042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.418539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 956/1907] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:06 2022] DELETE /identity/v3/projects/ba12976666de475194a11697e4183b97 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:07.419491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:07.420912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:07.503911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4e0fc-f083-4299-80a0-7a11333e92d9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:07.509489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:07.510338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:07.510815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:07.511076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] PATH_INFO: `/v3/projects/ed5cf5f74e394b4aa0a1a4d3daff7a12` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:07.513351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed5cf5f74e394b4aa0a1a4d3daff7a12)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:07.515607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:07.639950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b7c0078ecc441e6b6a1392ab8a1c528'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.640627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b7c0078ecc441e6b6a1392ab8a1c528'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.760173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed5cf5f74e394b4aa0a1a4d3daff7a12', 'request_id': 'req-59758d47-22b5-47da-b452-57cccb0751c7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.761418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b7c0078ecc441e6b6a1392ab8a1c528'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.766280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed5cf5f74e394b4aa0a1a4d3daff7a12', 'request_id': 'req-59758d47-22b5-47da-b452-57cccb0751c7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.782806 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b7c0078ecc441e6b6a1392ab8a1c528'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.783401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b7c0078ecc441e6b6a1392ab8a1c528'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.798239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 952/1908] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:07 2022] DELETE /identity/v3/users/4b7c0078ecc441e6b6a1392ab8a1c528 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:07.799365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:07.800969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:07.870527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b386c-c51f-461d-99ba-5e564821eff8 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:07.872883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:07.875463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99346a74-3bcb-4382-80b6-bcab8888ee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:07.876129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99346a74-3bcb-4382-80b6-bcab8888ee7b admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:07.876410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99346a74-3bcb-4382-80b6-bcab8888ee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:07.876804 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99346a74-3bcb-4382-80b6-bcab8888ee7b admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:07.877312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99346a74-3bcb-4382-80b6-bcab8888ee7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:07.879265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99346a74-3bcb-4382-80b6-bcab8888ee7b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:07.908960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed5cf5f74e394b4aa0a1a4d3daff7a12', 'request_id': 'req-59758d47-22b5-47da-b452-57cccb0751c7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.928809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] The token cache is being invalidate because project ed5cf5f74e394b4aa0a1a4d3daff7a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:07.929561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.930741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:07.939647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 957/1909] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:07 2022] DELETE /identity/v3/projects/ed5cf5f74e394b4aa0a1a4d3daff7a12 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:07.939647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:07.939647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:07.982620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 953/1910] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:07 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:07.984858 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63107b87-5a94-4e12-b4f3-a7817342ff76 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:07.985321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63107b87-5a94-4e12-b4f3-a7817342ff76 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:07.986240 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63107b87-5a94-4e12-b4f3-a7817342ff76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.011794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-63107b87-5a94-4e12-b4f3-a7817342ff76 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:08.013209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-63107b87-5a94-4e12-b4f3-a7817342ff76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:08.021388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59758d47-22b5-47da-b452-57cccb0751c7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:08.030492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 954/1911] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:08.030492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410e17-c988-40e3-8b59-9d08c45bff5e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.030492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410e17-c988-40e3-8b59-9d08c45bff5e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.030492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410e17-c988-40e3-8b59-9d08c45bff5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.037341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:08.042468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.042468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.042468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] PATH_INFO: `/v3/users/7656c3c3ebef4baa90c541695dbef060` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.042468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7656c3c3ebef4baa90c541695dbef060)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:08.051601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:08.075865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1d410e17-c988-40e3-8b59-9d08c45bff5e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:08.077880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1d410e17-c988-40e3-8b59-9d08c45bff5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:08.104638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 955/1912] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:08.104638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410e17-c988-40e3-8b59-9d08c45bff5e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:08.104638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1d410e17-c988-40e3-8b59-9d08c45bff5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:08.118596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410e17-c988-40e3-8b59-9d08c45bff5e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:08.125890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621c10d-28c8-4393-bc8b-28637b7c0663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:08.126868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621c10d-28c8-4393-bc8b-28637b7c0663 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.127359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621c10d-28c8-4393-bc8b-28637b7c0663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.127858 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621c10d-28c8-4393-bc8b-28637b7c0663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.128949 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621c10d-28c8-4393-bc8b-28637b7c0663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:08.132406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621c10d-28c8-4393-bc8b-28637b7c0663 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:08.152201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 956/1913] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:08.153874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621c10d-28c8-4393-bc8b-28637b7c0663 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:08.158312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c621c10d-28c8-4393-bc8b-28637b7c0663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:08.174243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:08.243066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621c10d-28c8-4393-bc8b-28637b7c0663 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:08.248793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b478d3-f4e0-4f45-9d17-67a822361c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:08.249798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b478d3-f4e0-4f45-9d17-67a822361c5b service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.250290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b478d3-f4e0-4f45-9d17-67a822361c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.250790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b478d3-f4e0-4f45-9d17-67a822361c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.262406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b478d3-f4e0-4f45-9d17-67a822361c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:08.263811 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b478d3-f4e0-4f45-9d17-67a822361c5b service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:08.274072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 957/1914] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:08.275191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775e3e7-0cbe-4d3e-baa8-97fcea1d96e0 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.275788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775e3e7-0cbe-4d3e-baa8-97fcea1d96e0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.275951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775e3e7-0cbe-4d3e-baa8-97fcea1d96e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.297067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7656c3c3ebef4baa90c541695dbef060'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:08.297367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8775e3e7-0cbe-4d3e-baa8-97fcea1d96e0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:08.298150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7656c3c3ebef4baa90c541695dbef060'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:08.298516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8775e3e7-0cbe-4d3e-baa8-97fcea1d96e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:08.312625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 958/1915] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:08.313402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775e3e7-0cbe-4d3e-baa8-97fcea1d96e0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:08.314920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8775e3e7-0cbe-4d3e-baa8-97fcea1d96e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:08.329473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775e3e7-0cbe-4d3e-baa8-97fcea1d96e0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:08.334463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587d8b7-2a9e-478b-a43c-ae0e2eb8741d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:08.335307 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587d8b7-2a9e-478b-a43c-ae0e2eb8741d admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.335618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587d8b7-2a9e-478b-a43c-ae0e2eb8741d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.335886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587d8b7-2a9e-478b-a43c-ae0e2eb8741d admin admin] PATH_INFO: `/v3/projects/ce76aea8290c48c4b08efc3e145bc7d6/users/ad559674fcc44ff9a4f3b00ad95c55a7/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.354969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587d8b7-2a9e-478b-a43c-ae0e2eb8741d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce76aea8290c48c4b08efc3e145bc7d6, user_id=ad559674fcc44ff9a4f3b00ad95c55a7, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:08.360917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587d8b7-2a9e-478b-a43c-ae0e2eb8741d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:08.408585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7656c3c3ebef4baa90c541695dbef060'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:08.426276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7656c3c3ebef4baa90c541695dbef060'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:08.428689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7656c3c3ebef4baa90c541695dbef060'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:08.446214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 958/1916] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:07 2022] DELETE /identity/v3/users/7656c3c3ebef4baa90c541695dbef060 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:08.447184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:08.460500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 959/1917] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:08 2022] PUT /identity/v3/projects/ce76aea8290c48c4b08efc3e145bc7d6/users/ad559674fcc44ff9a4f3b00ad95c55a7/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:08.466439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587d8b7-2a9e-478b-a43c-ae0e2eb8741d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:08.541659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587d8b7-2a9e-478b-a43c-ae0e2eb8741d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:08.546791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeca464-ba5d-4793-acef-8c252ee81536 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:08.553754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabca1d-9d2e-48d2-b657-95b9d4b15c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:08.554618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabca1d-9d2e-48d2-b657-95b9d4b15c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.554759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6090c0-3794-493c-b69d-0b622b7fbb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:08.555179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabca1d-9d2e-48d2-b657-95b9d4b15c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.555503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6090c0-3794-493c-b69d-0b622b7fbb43 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.556146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6090c0-3794-493c-b69d-0b622b7fbb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.556146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6090c0-3794-493c-b69d-0b622b7fbb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.556499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabca1d-9d2e-48d2-b657-95b9d4b15c78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.556785 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6090c0-3794-493c-b69d-0b622b7fbb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:08.557527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabca1d-9d2e-48d2-b657-95b9d4b15c78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:08.557973 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabca1d-9d2e-48d2-b657-95b9d4b15c78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:08.558677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6090c0-3794-493c-b69d-0b622b7fbb43 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:08.559949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabca1d-9d2e-48d2-b657-95b9d4b15c78 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:08.572375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 960/1918] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:08.578018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6090c0-3794-493c-b69d-0b622b7fbb43 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:08.579162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 959/1919] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:08.585492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabca1d-9d2e-48d2-b657-95b9d4b15c78 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:08.589347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6090c0-3794-493c-b69d-0b622b7fbb43 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:08.594514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac109c5-bdf9-4b24-b004-ca7a7ae0cb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:08.595330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac109c5-bdf9-4b24-b004-ca7a7ae0cb46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.595685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac109c5-bdf9-4b24-b004-ca7a7ae0cb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.595966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac109c5-bdf9-4b24-b004-ca7a7ae0cb46 admin admin] PATH_INFO: `/v3/projects/ce76aea8290c48c4b08efc3e145bc7d6/users/ad559674fcc44ff9a4f3b00ad95c55a7/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.596576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabca1d-9d2e-48d2-b657-95b9d4b15c78 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:08.598106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac109c5-bdf9-4b24-b004-ca7a7ae0cb46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce76aea8290c48c4b08efc3e145bc7d6, user_id=ad559674fcc44ff9a4f3b00ad95c55a7, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:08.602063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac109c5-bdf9-4b24-b004-ca7a7ae0cb46 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:08.608280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6246ad8-ea5c-4818-bf70-220b5162aedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:08.610379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6246ad8-ea5c-4818-bf70-220b5162aedf admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.614620 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6246ad8-ea5c-4818-bf70-220b5162aedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.614620 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6246ad8-ea5c-4818-bf70-220b5162aedf admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.614620 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6246ad8-ea5c-4818-bf70-220b5162aedf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:08.616102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 961/1920] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:08 2022] PUT /identity/v3/projects/ce76aea8290c48c4b08efc3e145bc7d6/users/ad559674fcc44ff9a4f3b00ad95c55a7/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:08.618266 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6246ad8-ea5c-4818-bf70-220b5162aedf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:08.634019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f2f22-4214-4a1a-a541-9ec6f7a60eeb None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:08.634593 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f2f22-4214-4a1a-a541-9ec6f7a60eeb None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:08.634882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f2f22-4214-4a1a-a541-9ec6f7a60eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:08.645661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:08.707124 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-827f2f22-4214-4a1a-a541-9ec6f7a60eeb None None] MFA Rules not processed for user `ad559674fcc44ff9a4f3b00ad95c55a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:08.708776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-827f2f22-4214-4a1a-a541-9ec6f7a60eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:08.847860 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 960/1921] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:08 2022] POST /identity/v3/projects => generated 383 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:08.858392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6246ad8-ea5c-4818-bf70-220b5162aedf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:08.858747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 962/1922] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:08.873391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f2f22-4214-4a1a-a541-9ec6f7a60eeb None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:08.877515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-827f2f22-4214-4a1a-a541-9ec6f7a60eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:09.000735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6246ad8-ea5c-4818-bf70-220b5162aedf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:09.012938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23905b24-1968-4f67-8d1d-49f1d640e690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:09.014567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23905b24-1968-4f67-8d1d-49f1d640e690 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:09.015090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23905b24-1968-4f67-8d1d-49f1d640e690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:09.015629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23905b24-1968-4f67-8d1d-49f1d640e690 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:09.017077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23905b24-1968-4f67-8d1d-49f1d640e690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:09.017262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f2f22-4214-4a1a-a541-9ec6f7a60eeb None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:09.027192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23905b24-1968-4f67-8d1d-49f1d640e690 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:09.103872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13516766-1e9e-42b3-a88a-cd9c468ee7dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:09.104794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13516766-1e9e-42b3-a88a-cd9c468ee7dc service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:09.104992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13516766-1e9e-42b3-a88a-cd9c468ee7dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:09.105276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13516766-1e9e-42b3-a88a-cd9c468ee7dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:09.187815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 961/1923] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:08 2022] POST /identity/v3/users => generated 292 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:09.187815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23905b24-1968-4f67-8d1d-49f1d640e690 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:09.187815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-23905b24-1968-4f67-8d1d-49f1d640e690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:09.208418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23905b24-1968-4f67-8d1d-49f1d640e690 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:09.214035 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:09.214879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:09.215449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:09.225427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] PATH_INFO: `/v3/projects/bade450ba43440f58c22858d8e1af7e1` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:09.225427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bade450ba43440f58c22858d8e1af7e1)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:09.225427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:09.225730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13516766-1e9e-42b3-a88a-cd9c468ee7dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:09.229092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13516766-1e9e-42b3-a88a-cd9c468ee7dc service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:09.252751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 963/1924] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:30:09.254273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13516766-1e9e-42b3-a88a-cd9c468ee7dc service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:09.257061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-13516766-1e9e-42b3-a88a-cd9c468ee7dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:09.284336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13516766-1e9e-42b3-a88a-cd9c468ee7dc service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:09.296417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:09.296417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:09.296417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:09.296417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] PATH_INFO: `/v3/projects/7a9ebe81aa324299aa386a2e2ffbf6e8` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:09.298817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9ebe81aa324299aa386a2e2ffbf6e8)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:09.308655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:09.411474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bade450ba43440f58c22858d8e1af7e1', 'request_id': 'req-a2304ea7-e5ea-4293-946e-f9e653aee271'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.418144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bade450ba43440f58c22858d8e1af7e1', 'request_id': 'req-a2304ea7-e5ea-4293-946e-f9e653aee271'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.479130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9ebe81aa324299aa386a2e2ffbf6e8', 'request_id': 'req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.485749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9ebe81aa324299aa386a2e2ffbf6e8', 'request_id': 'req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.537425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bade450ba43440f58c22858d8e1af7e1', 'request_id': 'req-a2304ea7-e5ea-4293-946e-f9e653aee271'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.552499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] The token cache is being invalidate because project bade450ba43440f58c22858d8e1af7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:09.552833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.553648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.555586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 962/1925] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:09 2022] DELETE /identity/v3/projects/bade450ba43440f58c22858d8e1af7e1 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:09.556929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:09.565142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:09.617196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9ebe81aa324299aa386a2e2ffbf6e8', 'request_id': 'req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.630777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] The token cache is being invalidate because project 7a9ebe81aa324299aa386a2e2ffbf6e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:09.631057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.631751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.632892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2304ea7-e5ea-4293-946e-f9e653aee271 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:09.633304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 964/1926] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:09 2022] DELETE /identity/v3/projects/7a9ebe81aa324299aa386a2e2ffbf6e8 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:09.634435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:09.635773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:09.696029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154f3f9-84d1-4b69-ab02-5a7dc3b71782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:09.697045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154f3f9-84d1-4b69-ab02-5a7dc3b71782 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:09.697502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154f3f9-84d1-4b69-ab02-5a7dc3b71782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:09.697992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154f3f9-84d1-4b69-ab02-5a7dc3b71782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:09.699089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154f3f9-84d1-4b69-ab02-5a7dc3b71782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:09.701945 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154f3f9-84d1-4b69-ab02-5a7dc3b71782 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:09.710819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7eb065-d9f6-470d-a1a7-266aa8f3e7a5 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:09.719986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 963/1927] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:09.721765 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154f3f9-84d1-4b69-ab02-5a7dc3b71782 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:09.724555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1154f3f9-84d1-4b69-ab02-5a7dc3b71782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:09.726467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:09.727125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:09.727402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:09.727719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] PATH_INFO: `/v3/users/2d6bad4ac24f41db8b30dc42cf8b97f9` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:09.728942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d6bad4ac24f41db8b30dc42cf8b97f9)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:09.731005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:09.739308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154f3f9-84d1-4b69-ab02-5a7dc3b71782 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:09.745545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:09.746235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:09.746518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:09.746792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] PATH_INFO: `/v3/users/4f143567da3946c5a518e3197b4a303d` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:09.748024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f143567da3946c5a518e3197b4a303d)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:09.749914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:09.936369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d6bad4ac24f41db8b30dc42cf8b97f9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:09.936994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d6bad4ac24f41db8b30dc42cf8b97f9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:10.001199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d6bad4ac24f41db8b30dc42cf8b97f9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:10.027568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f143567da3946c5a518e3197b4a303d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:10.029707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f143567da3946c5a518e3197b4a303d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:10.042748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d6bad4ac24f41db8b30dc42cf8b97f9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:10.043783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-74a9d1e3-d2f4-4282-bb08-4753caba75c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d6bad4ac24f41db8b30dc42cf8b97f9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:10.069190 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 965/1928] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:09 2022] DELETE /identity/v3/users/2d6bad4ac24f41db8b30dc42cf8b97f9 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:10.070310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aee98-394a-4f11-9f1a-155eae841875 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.070747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aee98-394a-4f11-9f1a-155eae841875 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.071162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aee98-394a-4f11-9f1a-155eae841875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.107947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f143567da3946c5a518e3197b4a303d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:10.109042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ca3aee98-394a-4f11-9f1a-155eae841875 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:10.111385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ca3aee98-394a-4f11-9f1a-155eae841875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:10.132756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f143567da3946c5a518e3197b4a303d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:10.133507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f143567da3946c5a518e3197b4a303d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:10.154034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 964/1929] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:09 2022] DELETE /identity/v3/users/4f143567da3946c5a518e3197b4a303d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:10.157580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:10.160825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:10.214463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:10.247434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 966/1930] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:10.247434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12f63d-de3d-4042-8fb6-e8d3a077d296 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.247434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12f63d-de3d-4042-8fb6-e8d3a077d296 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.247434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12f63d-de3d-4042-8fb6-e8d3a077d296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.268840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1e12f63d-de3d-4042-8fb6-e8d3a077d296 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:10.270311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1e12f63d-de3d-4042-8fb6-e8d3a077d296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:10.323399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a835a7-db16-4aed-917e-cfa0e4ee73f5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:10.331075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca583433-e4e7-4127-9418-698a62dcd793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:10.331748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca583433-e4e7-4127-9418-698a62dcd793 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.332033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca583433-e4e7-4127-9418-698a62dcd793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.333317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca583433-e4e7-4127-9418-698a62dcd793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.333508 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca583433-e4e7-4127-9418-698a62dcd793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:10.335167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca583433-e4e7-4127-9418-698a62dcd793 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:10.349091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 967/1931] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:10.349351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 965/1932] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:10.350466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca583433-e4e7-4127-9418-698a62dcd793 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:10.350931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250c6e7-10ae-4021-8e15-b422d8ebc426 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.351284 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250c6e7-10ae-4021-8e15-b422d8ebc426 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.351702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250c6e7-10ae-4021-8e15-b422d8ebc426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.363193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca583433-e4e7-4127-9418-698a62dcd793 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:10.369580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f23222-5874-4f41-8861-464b71215d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:10.372825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f23222-5874-4f41-8861-464b71215d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.372825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f23222-5874-4f41-8861-464b71215d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.372825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f23222-5874-4f41-8861-464b71215d7c admin admin] PATH_INFO: `/v3/projects/6f6fb474a92c4dc78d377e1adf1cc8cb/users/8125ec2ad31b45b19b7d831fc11c8d26/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.375398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-4250c6e7-10ae-4021-8e15-b422d8ebc426 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:10.377326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4250c6e7-10ae-4021-8e15-b422d8ebc426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:10.390393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 968/1933] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:10.390899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f23222-5874-4f41-8861-464b71215d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6fb474a92c4dc78d377e1adf1cc8cb, user_id=8125ec2ad31b45b19b7d831fc11c8d26, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:10.392064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250c6e7-10ae-4021-8e15-b422d8ebc426 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:10.393911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4250c6e7-10ae-4021-8e15-b422d8ebc426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:10.396062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f23222-5874-4f41-8861-464b71215d7c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:10.487815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250c6e7-10ae-4021-8e15-b422d8ebc426 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:10.493923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 966/1934] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:10 2022] PUT /identity/v3/projects/6f6fb474a92c4dc78d377e1adf1cc8cb/users/8125ec2ad31b45b19b7d831fc11c8d26/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:10.496851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f23222-5874-4f41-8861-464b71215d7c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:10.562799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:10.585365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12820fbb-c85a-4ac3-84d3-bc36bf384013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:10.585994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12820fbb-c85a-4ac3-84d3-bc36bf384013 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.586833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12820fbb-c85a-4ac3-84d3-bc36bf384013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.587160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12820fbb-c85a-4ac3-84d3-bc36bf384013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.714455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f23222-5874-4f41-8861-464b71215d7c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:10.728964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242c16d-b54e-4522-bdd3-b084ba3b377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:10.729514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242c16d-b54e-4522-bdd3-b084ba3b377e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.730291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242c16d-b54e-4522-bdd3-b084ba3b377e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.730669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242c16d-b54e-4522-bdd3-b084ba3b377e admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.731433 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242c16d-b54e-4522-bdd3-b084ba3b377e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:10.731860 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242c16d-b54e-4522-bdd3-b084ba3b377e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:10.734058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242c16d-b54e-4522-bdd3-b084ba3b377e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:10.734364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12820fbb-c85a-4ac3-84d3-bc36bf384013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:10.740412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12820fbb-c85a-4ac3-84d3-bc36bf384013 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:10.748755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 967/1935] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:10.750777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25226162-80e7-4db5-8a24-c628ebee8b7c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.751379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25226162-80e7-4db5-8a24-c628ebee8b7c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.751873 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25226162-80e7-4db5-8a24-c628ebee8b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.761464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 969/1936] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:10.762347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12820fbb-c85a-4ac3-84d3-bc36bf384013 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:10.773362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12820fbb-c85a-4ac3-84d3-bc36bf384013 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:10.778414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512ac5b-b910-4c2a-b4d0-76f6a076c856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:10.779032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512ac5b-b910-4c2a-b4d0-76f6a076c856 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.779320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512ac5b-b910-4c2a-b4d0-76f6a076c856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.779699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512ac5b-b910-4c2a-b4d0-76f6a076c856 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.780352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512ac5b-b910-4c2a-b4d0-76f6a076c856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:10.782291 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512ac5b-b910-4c2a-b4d0-76f6a076c856 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:10.787600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-25226162-80e7-4db5-8a24-c628ebee8b7c None None] MFA Rules not processed for user `8125ec2ad31b45b19b7d831fc11c8d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:10.788995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-25226162-80e7-4db5-8a24-c628ebee8b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:10.859470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 968/1937] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:10.863960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25226162-80e7-4db5-8a24-c628ebee8b7c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:10.882259 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 970/1938] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:10 2022] POST /identity/v3/projects => generated 393 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:10.885094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512ac5b-b910-4c2a-b4d0-76f6a076c856 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:10.896403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25226162-80e7-4db5-8a24-c628ebee8b7c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:10.962278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf5eb5-35c8-4597-9e35-a43a5eb94414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:10.963127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf5eb5-35c8-4597-9e35-a43a5eb94414 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.963560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf5eb5-35c8-4597-9e35-a43a5eb94414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.963948 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf5eb5-35c8-4597-9e35-a43a5eb94414 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.964920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf5eb5-35c8-4597-9e35-a43a5eb94414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:10.985628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf5eb5-35c8-4597-9e35-a43a5eb94414 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:10.986993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512ac5b-b910-4c2a-b4d0-76f6a076c856 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:10.993806 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e05a04-d938-4bbc-b7f6-efcdc43df6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:10.995178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e05a04-d938-4bbc-b7f6-efcdc43df6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:10.995735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e05a04-d938-4bbc-b7f6-efcdc43df6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:10.996207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e05a04-d938-4bbc-b7f6-efcdc43df6b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:10.997691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e05a04-d938-4bbc-b7f6-efcdc43df6b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:11.000418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e05a04-d938-4bbc-b7f6-efcdc43df6b2 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:11.225694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 971/1939] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:10 2022] POST /identity/v3/users => generated 297 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:11.226452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e05a04-d938-4bbc-b7f6-efcdc43df6b2 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:11.227084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 969/1940] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:10 2022] POST /identity/v3/projects => generated 383 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:11.229448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b5e05a04-d938-4bbc-b7f6-efcdc43df6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:11.230311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf5eb5-35c8-4597-9e35-a43a5eb94414 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:11.244590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e05a04-d938-4bbc-b7f6-efcdc43df6b2 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:11.331701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:11.332346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:11.333151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:11.333641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] PATH_INFO: `/v3/projects/e5372c3103f544ccaccb6ab3be83f16c` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:11.334803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5372c3103f544ccaccb6ab3be83f16c)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:11.337419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:11.357358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf5eb5-35c8-4597-9e35-a43a5eb94414 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:11.362815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89083c8-2b3d-4921-8324-143ed222d257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:11.363561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89083c8-2b3d-4921-8324-143ed222d257 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:11.363887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89083c8-2b3d-4921-8324-143ed222d257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:11.364162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89083c8-2b3d-4921-8324-143ed222d257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:11.364808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89083c8-2b3d-4921-8324-143ed222d257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:11.366415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89083c8-2b3d-4921-8324-143ed222d257 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:11.377564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 970/1941] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:11.378281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89083c8-2b3d-4921-8324-143ed222d257 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:11.392731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89083c8-2b3d-4921-8324-143ed222d257 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:11.405136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466b5c5-bd4e-4647-832a-691e686b188c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:11.405136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466b5c5-bd4e-4647-832a-691e686b188c admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:11.405136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466b5c5-bd4e-4647-832a-691e686b188c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:11.405136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466b5c5-bd4e-4647-832a-691e686b188c admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:11.405517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466b5c5-bd4e-4647-832a-691e686b188c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:11.407420 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466b5c5-bd4e-4647-832a-691e686b188c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:11.513203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5372c3103f544ccaccb6ab3be83f16c', 'request_id': 'req-f67aac04-4388-4747-82c7-683da3e2312c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:11.523636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5372c3103f544ccaccb6ab3be83f16c', 'request_id': 'req-f67aac04-4388-4747-82c7-683da3e2312c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:11.532168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 971/1942] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:11 2022] POST /identity/v3/users => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:11.535177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466b5c5-bd4e-4647-832a-691e686b188c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:11.558471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466b5c5-bd4e-4647-832a-691e686b188c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:11.626975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5372c3103f544ccaccb6ab3be83f16c', 'request_id': 'req-f67aac04-4388-4747-82c7-683da3e2312c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:11.651185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] The token cache is being invalidate because project e5372c3103f544ccaccb6ab3be83f16c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:11.651185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:11.652725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:11.655276 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 972/1943] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:11 2022] DELETE /identity/v3/projects/e5372c3103f544ccaccb6ab3be83f16c => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:11.657531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:11.660603 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:11.701774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc40cf-fcc0-4076-851d-6ade3f50fd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:11.702722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc40cf-fcc0-4076-851d-6ade3f50fd51 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:11.703159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc40cf-fcc0-4076-851d-6ade3f50fd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:11.703648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc40cf-fcc0-4076-851d-6ade3f50fd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:11.704574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc40cf-fcc0-4076-851d-6ade3f50fd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:11.706463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc40cf-fcc0-4076-851d-6ade3f50fd51 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:11.731430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 972/1944] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:11.731430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc40cf-fcc0-4076-851d-6ade3f50fd51 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:11.733552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c1cc40cf-fcc0-4076-851d-6ade3f50fd51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:11.759270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc40cf-fcc0-4076-851d-6ade3f50fd51 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:11.784573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:11.795451 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:11.796768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:11.796768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:11.796768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] PATH_INFO: `/v3/users/0284c828e29f41aa86bb4c20e5bc0392` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:11.798008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0284c828e29f41aa86bb4c20e5bc0392)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:11.801036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:11.801934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aac04-4388-4747-82c7-683da3e2312c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:11.807252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1fb22-f3c2-4361-aeee-2ef9aac4a677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:11.807907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1fb22-f3c2-4361-aeee-2ef9aac4a677 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:11.812693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1fb22-f3c2-4361-aeee-2ef9aac4a677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:11.812693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1fb22-f3c2-4361-aeee-2ef9aac4a677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:11.812693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1fb22-f3c2-4361-aeee-2ef9aac4a677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:11.812693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1fb22-f3c2-4361-aeee-2ef9aac4a677 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:11.822917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 973/1945] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:11.824208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1fb22-f3c2-4361-aeee-2ef9aac4a677 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:11.826093 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-87b1fb22-f3c2-4361-aeee-2ef9aac4a677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:11.840676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1fb22-f3c2-4361-aeee-2ef9aac4a677 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:11.846091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcff5bc-22dd-42c7-9f31-16b27cb6277a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:11.846872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff5bc-22dd-42c7-9f31-16b27cb6277a admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:11.847313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff5bc-22dd-42c7-9f31-16b27cb6277a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:11.847707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff5bc-22dd-42c7-9f31-16b27cb6277a admin admin] PATH_INFO: `/v3/projects/8c8ab2cb4bc8454f8c99dd03c5b6608e/users/5ade72135cbf4cbda435722e759e3892/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:11.865867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcff5bc-22dd-42c7-9f31-16b27cb6277a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c8ab2cb4bc8454f8c99dd03c5b6608e, user_id=5ade72135cbf4cbda435722e759e3892, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:11.868712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcff5bc-22dd-42c7-9f31-16b27cb6277a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:11.976551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 974/1946] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:11 2022] PUT /identity/v3/projects/8c8ab2cb4bc8454f8c99dd03c5b6608e/users/5ade72135cbf4cbda435722e759e3892/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:11.991222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcff5bc-22dd-42c7-9f31-16b27cb6277a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:11.994293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2fcff5bc-22dd-42c7-9f31-16b27cb6277a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:12.091649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0284c828e29f41aa86bb4c20e5bc0392'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:12.095021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0284c828e29f41aa86bb4c20e5bc0392'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:12.163803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcff5bc-22dd-42c7-9f31-16b27cb6277a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:12.182339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:12.182339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:12.182984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:12.183494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] PATH_INFO: `/v3/projects/38c92e2b51b2477fb181f731e7c412c7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:12.184732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0284c828e29f41aa86bb4c20e5bc0392'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:12.186489 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c92e2b51b2477fb181f731e7c412c7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:12.189473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:12.204099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0284c828e29f41aa86bb4c20e5bc0392'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:12.207023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0284c828e29f41aa86bb4c20e5bc0392'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:12.227172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 973/1947] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:11 2022] DELETE /identity/v3/users/0284c828e29f41aa86bb4c20e5bc0392 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:12.229454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:12.230969 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:12.381605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c92e2b51b2477fb181f731e7c412c7', 'request_id': 'req-907c2631-291b-4cb3-b98a-33ba331e209e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:12.404087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c92e2b51b2477fb181f731e7c412c7', 'request_id': 'req-907c2631-291b-4cb3-b98a-33ba331e209e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:12.420054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5dc7a-ea92-4e01-b2d8-fe2f4d8086c7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:12.507052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c92e2b51b2477fb181f731e7c412c7', 'request_id': 'req-907c2631-291b-4cb3-b98a-33ba331e209e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:12.528492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] The token cache is being invalidate because project 38c92e2b51b2477fb181f731e7c412c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:12.528899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:12.530290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-907c2631-291b-4cb3-b98a-33ba331e209e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:12.538706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 975/1948] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:11 2022] DELETE /identity/v3/projects/38c92e2b51b2477fb181f731e7c412c7 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:12.540928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a6705-09cf-46ec-8e5b-2af9f1aed09e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:12.541627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a6705-09cf-46ec-8e5b-2af9f1aed09e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:12.541901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a6705-09cf-46ec-8e5b-2af9f1aed09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:12.549759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dcad4-6dbd-4531-a69c-8e6b0739c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:12.551697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dcad4-6dbd-4531-a69c-8e6b0739c7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:12.552294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dcad4-6dbd-4531-a69c-8e6b0739c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:12.552806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dcad4-6dbd-4531-a69c-8e6b0739c7c1 admin admin] PATH_INFO: `/v3/projects/3568f2075b4d4e5981741ebd48c20fa9/users/46177fa6653a4d98b6f01a2e4c5e4764/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:12.575741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0dcad4-6dbd-4531-a69c-8e6b0739c7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3568f2075b4d4e5981741ebd48c20fa9, user_id=46177fa6653a4d98b6f01a2e4c5e4764, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:12.578467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0dcad4-6dbd-4531-a69c-8e6b0739c7c1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:12.579370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-416a6705-09cf-46ec-8e5b-2af9f1aed09e None None] MFA Rules not processed for user `5ade72135cbf4cbda435722e759e3892`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:12.581323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-416a6705-09cf-46ec-8e5b-2af9f1aed09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:12.659194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 976/1949] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:12.660225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a6705-09cf-46ec-8e5b-2af9f1aed09e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:12.662851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-416a6705-09cf-46ec-8e5b-2af9f1aed09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:12.671655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 974/1950] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:12 2022] PUT /identity/v3/projects/3568f2075b4d4e5981741ebd48c20fa9/users/46177fa6653a4d98b6f01a2e4c5e4764/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:12.673946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dcad4-6dbd-4531-a69c-8e6b0739c7c1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:12.674895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0c0dcad4-6dbd-4531-a69c-8e6b0739c7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:12.682946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a6705-09cf-46ec-8e5b-2af9f1aed09e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:12.769848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:12.771308 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:12.771710 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:12.772202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] PATH_INFO: `/v3/users/ad559674fcc44ff9a4f3b00ad95c55a7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:12.774242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad559674fcc44ff9a4f3b00ad95c55a7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:12.776575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:12.827690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dcad4-6dbd-4531-a69c-8e6b0739c7c1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:12.837188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50744fc-e287-4afb-9ff4-38e89248d469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:12.837872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50744fc-e287-4afb-9ff4-38e89248d469 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:12.838468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50744fc-e287-4afb-9ff4-38e89248d469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:12.838839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50744fc-e287-4afb-9ff4-38e89248d469 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:12.839631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50744fc-e287-4afb-9ff4-38e89248d469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:12.842562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50744fc-e287-4afb-9ff4-38e89248d469 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:12.852664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:12.983168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 975/1951] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:12 2022] POST /identity/v3/projects => generated 391 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:12.985712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50744fc-e287-4afb-9ff4-38e89248d469 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:12.988696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e50744fc-e287-4afb-9ff4-38e89248d469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:13.082158 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad559674fcc44ff9a4f3b00ad95c55a7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.084832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad559674fcc44ff9a4f3b00ad95c55a7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.101877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50744fc-e287-4afb-9ff4-38e89248d469 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:13.107981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1f754-1915-4be7-bf0a-97b6a5d632ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:13.108965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1f754-1915-4be7-bf0a-97b6a5d632ae admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.109467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1f754-1915-4be7-bf0a-97b6a5d632ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.110073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1f754-1915-4be7-bf0a-97b6a5d632ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.110971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1f754-1915-4be7-bf0a-97b6a5d632ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:13.112996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1f754-1915-4be7-bf0a-97b6a5d632ae admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:13.127776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 976/1952] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:13.129106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1f754-1915-4be7-bf0a-97b6a5d632ae admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:13.132905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5bb1f754-1915-4be7-bf0a-97b6a5d632ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:13.156854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1f754-1915-4be7-bf0a-97b6a5d632ae admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:13.161999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:13.163082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.163562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.163949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] PATH_INFO: `/v3/projects/db29621521a14d16b05afc7ceb841d8e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.165550 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=db29621521a14d16b05afc7ceb841d8e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:13.167683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:13.172233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:13.223252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad559674fcc44ff9a4f3b00ad95c55a7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.237789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad559674fcc44ff9a4f3b00ad95c55a7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.238527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad559674fcc44ff9a4f3b00ad95c55a7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.257524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 977/1953] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:12 2022] DELETE /identity/v3/users/ad559674fcc44ff9a4f3b00ad95c55a7 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:13.258577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:13.260090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:13.308033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db29621521a14d16b05afc7ceb841d8e', 'request_id': 'req-7b16c38a-db95-4f86-9d1b-4191b09897cc'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.316152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db29621521a14d16b05afc7ceb841d8e', 'request_id': 'req-7b16c38a-db95-4f86-9d1b-4191b09897cc'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.352667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdf479-4033-415c-912d-c47cd85ed3d1 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:13.430853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1746460-c5d2-4e48-8c0c-45d1bb59c5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:13.431701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1746460-c5d2-4e48-8c0c-45d1bb59c5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.432076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1746460-c5d2-4e48-8c0c-45d1bb59c5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.432353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1746460-c5d2-4e48-8c0c-45d1bb59c5e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.433251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1746460-c5d2-4e48-8c0c-45d1bb59c5e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:13.435582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1746460-c5d2-4e48-8c0c-45d1bb59c5e1 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:13.448938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db29621521a14d16b05afc7ceb841d8e', 'request_id': 'req-7b16c38a-db95-4f86-9d1b-4191b09897cc'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.470303 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] The token cache is being invalidate because project db29621521a14d16b05afc7ceb841d8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:13.471384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.472550 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:13.474982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 977/1954] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:13 2022] DELETE /identity/v3/projects/db29621521a14d16b05afc7ceb841d8e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:13.476195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:13.477990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:13.503380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16c38a-db95-4f86-9d1b-4191b09897cc admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:13.510321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404ebba-e0e2-4856-aebb-fd409b3d7cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:13.511589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ebba-e0e2-4856-aebb-fd409b3d7cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.512039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ebba-e0e2-4856-aebb-fd409b3d7cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.512632 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ebba-e0e2-4856-aebb-fd409b3d7cb9 admin admin] PATH_INFO: `/v3/projects/3568f2075b4d4e5981741ebd48c20fa9/users/46177fa6653a4d98b6f01a2e4c5e4764/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.515114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404ebba-e0e2-4856-aebb-fd409b3d7cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3568f2075b4d4e5981741ebd48c20fa9, user_id=46177fa6653a4d98b6f01a2e4c5e4764, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:13.528028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404ebba-e0e2-4856-aebb-fd409b3d7cb9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:13.544498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 978/1955] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:13 2022] PUT /identity/v3/projects/3568f2075b4d4e5981741ebd48c20fa9/users/46177fa6653a4d98b6f01a2e4c5e4764/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:13.546465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f4982-0364-4bce-9248-3f7d05cbea5c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.546822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f4982-0364-4bce-9248-3f7d05cbea5c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.547265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f4982-0364-4bce-9248-3f7d05cbea5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.570602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 978/1956] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:13 2022] POST /identity/v3/users => generated 302 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:13.570602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882e202-134c-44a6-93f7-7b74d02abdbf None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.570602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882e202-134c-44a6-93f7-7b74d02abdbf None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.570602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882e202-134c-44a6-93f7-7b74d02abdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.597651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-014f4982-0364-4bce-9248-3f7d05cbea5c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:13.602115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-014f4982-0364-4bce-9248-3f7d05cbea5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:13.666109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1882e202-134c-44a6-93f7-7b74d02abdbf None None] MFA Rules not processed for user `46177fa6653a4d98b6f01a2e4c5e4764`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:13.670520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1882e202-134c-44a6-93f7-7b74d02abdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:13.795706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 979/1957] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:13.796576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f4982-0364-4bce-9248-3f7d05cbea5c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:13.798726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-014f4982-0364-4bce-9248-3f7d05cbea5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:13.826280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f4982-0364-4bce-9248-3f7d05cbea5c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:13.844144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad6c4b-d91c-45b8-8b94-246d82ba5017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:13.845969 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad6c4b-d91c-45b8-8b94-246d82ba5017 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.846245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad6c4b-d91c-45b8-8b94-246d82ba5017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.846513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad6c4b-d91c-45b8-8b94-246d82ba5017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.849779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad6c4b-d91c-45b8-8b94-246d82ba5017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:13.855136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad6c4b-d91c-45b8-8b94-246d82ba5017 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:13.866687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 979/1958] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:13.868657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac7dc1-90ce-4e73-b73a-4888c3533144 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.869012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac7dc1-90ce-4e73-b73a-4888c3533144 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.869343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac7dc1-90ce-4e73-b73a-4888c3533144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.882247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 980/1959] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:13.885532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad6c4b-d91c-45b8-8b94-246d82ba5017 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:13.913209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad6c4b-d91c-45b8-8b94-246d82ba5017 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:13.917543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-cfac7dc1-90ce-4e73-b73a-4888c3533144 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:13.918836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cfac7dc1-90ce-4e73-b73a-4888c3533144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:13.920984 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a6e06-f397-45ec-be21-8ae8d691de2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:13.922586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a6e06-f397-45ec-be21-8ae8d691de2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.923456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a6e06-f397-45ec-be21-8ae8d691de2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.923922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a6e06-f397-45ec-be21-8ae8d691de2a admin admin] PATH_INFO: `/v3/projects/323fbde6748e4e559fef1fb27459fb8a/users/d4a51da848a4444393bcbdadbe9dcd94/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.937859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 980/1960] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:13.944851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac7dc1-90ce-4e73-b73a-4888c3533144 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:13.947815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a6e06-f397-45ec-be21-8ae8d691de2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=323fbde6748e4e559fef1fb27459fb8a, user_id=d4a51da848a4444393bcbdadbe9dcd94, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:13.950905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a6e06-f397-45ec-be21-8ae8d691de2a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:13.974292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac7dc1-90ce-4e73-b73a-4888c3533144 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:13.980084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af7186-ee77-49dc-955c-3819f0cd3138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:13.981088 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af7186-ee77-49dc-955c-3819f0cd3138 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:13.982252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af7186-ee77-49dc-955c-3819f0cd3138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:13.982770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af7186-ee77-49dc-955c-3819f0cd3138 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:13.983734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af7186-ee77-49dc-955c-3819f0cd3138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:13.984565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af7186-ee77-49dc-955c-3819f0cd3138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:13.987349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af7186-ee77-49dc-955c-3819f0cd3138 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.004656 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 981/1961] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:14.005579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af7186-ee77-49dc-955c-3819f0cd3138 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.007686 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-57af7186-ee77-49dc-955c-3819f0cd3138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:14.108745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af7186-ee77-49dc-955c-3819f0cd3138 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.120128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec72460-ba4e-43f1-9692-22f7ecdfa86b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.121003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec72460-ba4e-43f1-9692-22f7ecdfa86b service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.121454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec72460-ba4e-43f1-9692-22f7ecdfa86b service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.121903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec72460-ba4e-43f1-9692-22f7ecdfa86b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.150234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:14.153241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec72460-ba4e-43f1-9692-22f7ecdfa86b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.155060 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec72460-ba4e-43f1-9692-22f7ecdfa86b service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.164075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 981/1962] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:13 2022] PUT /identity/v3/projects/323fbde6748e4e559fef1fb27459fb8a/users/d4a51da848a4444393bcbdadbe9dcd94/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:14.164075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a6e06-f397-45ec-be21-8ae8d691de2a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.166481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-564a6e06-f397-45ec-be21-8ae8d691de2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:14.177270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 982/1963] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:14.178281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec72460-ba4e-43f1-9692-22f7ecdfa86b service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.207332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec72460-ba4e-43f1-9692-22f7ecdfa86b service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.352897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a6e06-f397-45ec-be21-8ae8d691de2a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.362341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934b77a-f438-404c-9179-654054c17a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.363454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934b77a-f438-404c-9179-654054c17a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.363704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.364775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.365195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.365608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] PATH_INFO: `/v3/projects/ce76aea8290c48c4b08efc3e145bc7d6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.366306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934b77a-f438-404c-9179-654054c17a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.366306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934b77a-f438-404c-9179-654054c17a4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.368530 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce76aea8290c48c4b08efc3e145bc7d6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.370383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934b77a-f438-404c-9179-654054c17a4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.373572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934b77a-f438-404c-9179-654054c17a4a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.375439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.510770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce76aea8290c48c4b08efc3e145bc7d6', 'request_id': 'req-e6b2652b-b866-431f-ad60-61e0f17fa1c4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:14.517335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce76aea8290c48c4b08efc3e145bc7d6', 'request_id': 'req-e6b2652b-b866-431f-ad60-61e0f17fa1c4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:14.572026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 983/1964] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:14 2022] POST /identity/v3/projects => generated 385 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:14.572026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934b77a-f438-404c-9179-654054c17a4a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.654520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934b77a-f438-404c-9179-654054c17a4a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.659879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fefd0-86c6-4a64-a2df-524e543fb09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.661023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fefd0-86c6-4a64-a2df-524e543fb09e admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.661023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fefd0-86c6-4a64-a2df-524e543fb09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.661464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fefd0-86c6-4a64-a2df-524e543fb09e admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.662238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fefd0-86c6-4a64-a2df-524e543fb09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.663983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fefd0-86c6-4a64-a2df-524e543fb09e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.675716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 984/1965] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:14.677034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fefd0-86c6-4a64-a2df-524e543fb09e admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.683393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce76aea8290c48c4b08efc3e145bc7d6', 'request_id': 'req-e6b2652b-b866-431f-ad60-61e0f17fa1c4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:14.708690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] The token cache is being invalidate because project ce76aea8290c48c4b08efc3e145bc7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:14.709023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:14.709848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:14.711621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 982/1966] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:14 2022] DELETE /identity/v3/projects/ce76aea8290c48c4b08efc3e145bc7d6 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:14.729339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.730895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:14.829755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fefd0-86c6-4a64-a2df-524e543fb09e admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.840088 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d913d1-7537-4598-b18e-311a5439c8c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.840806 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d913d1-7537-4598-b18e-311a5439c8c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.841157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d913d1-7537-4598-b18e-311a5439c8c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.841775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d913d1-7537-4598-b18e-311a5439c8c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.844762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c5d913d1-7537-4598-b18e-311a5439c8c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:14.862250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2652b-b866-431f-ad60-61e0f17fa1c4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.868777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f845f8-18b2-403a-9759-6289313aacc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.869996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f845f8-18b2-403a-9759-6289313aacc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.871176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f845f8-18b2-403a-9759-6289313aacc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.871809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f845f8-18b2-403a-9759-6289313aacc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.873570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f845f8-18b2-403a-9759-6289313aacc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.876013 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f845f8-18b2-403a-9759-6289313aacc4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.930573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d913d1-7537-4598-b18e-311a5439c8c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.932066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d913d1-7537-4598-b18e-311a5439c8c3 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:14.943149 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 985/1967] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:14.943911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d913d1-7537-4598-b18e-311a5439c8c3 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:14.945290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c5d913d1-7537-4598-b18e-311a5439c8c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:14.960887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d913d1-7537-4598-b18e-311a5439c8c3 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:14.966049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19738d4e-eec2-41d5-b5cc-dff285e2d9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:14.966738 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19738d4e-eec2-41d5-b5cc-dff285e2d9f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:14.967017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19738d4e-eec2-41d5-b5cc-dff285e2d9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:14.967297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19738d4e-eec2-41d5-b5cc-dff285e2d9f2 admin admin] PATH_INFO: `/v3/projects/323fbde6748e4e559fef1fb27459fb8a/users/d4a51da848a4444393bcbdadbe9dcd94/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:14.971285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19738d4e-eec2-41d5-b5cc-dff285e2d9f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=323fbde6748e4e559fef1fb27459fb8a, user_id=d4a51da848a4444393bcbdadbe9dcd94, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:14.974929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19738d4e-eec2-41d5-b5cc-dff285e2d9f2 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:15.005343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 983/1968] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:14 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:15.005343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c670daf-cf41-471f-bdcf-0655d6931200 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.005343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c670daf-cf41-471f-bdcf-0655d6931200 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.005343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c670daf-cf41-471f-bdcf-0655d6931200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.027575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 986/1969] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:14 2022] PUT /identity/v3/projects/323fbde6748e4e559fef1fb27459fb8a/users/d4a51da848a4444393bcbdadbe9dcd94/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:15.027575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19738d4e-eec2-41d5-b5cc-dff285e2d9f2 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:15.029302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:15.040047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8c670daf-cf41-471f-bdcf-0655d6931200 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:15.042194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8c670daf-cf41-471f-bdcf-0655d6931200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:15.127061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19738d4e-eec2-41d5-b5cc-dff285e2d9f2 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:15.136386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ed3ae-12ef-4723-b2c3-f2c4f1cc67a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:15.137494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ed3ae-12ef-4723-b2c3-f2c4f1cc67a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.138014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ed3ae-12ef-4723-b2c3-f2c4f1cc67a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.138698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ed3ae-12ef-4723-b2c3-f2c4f1cc67a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.146571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 984/1970] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:15.146571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8edd2-de3a-4181-8c65-798e42a2da7d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.146571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8edd2-de3a-4181-8c65-798e42a2da7d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.146571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8edd2-de3a-4181-8c65-798e42a2da7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.151761 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ed3ae-12ef-4723-b2c3-f2c4f1cc67a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:15.155837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ed3ae-12ef-4723-b2c3-f2c4f1cc67a9 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:15.184655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-caf8edd2-de3a-4181-8c65-798e42a2da7d None None] MFA Rules not processed for user `d4a51da848a4444393bcbdadbe9dcd94`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:15.184655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-caf8edd2-de3a-4181-8c65-798e42a2da7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:15.193620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 987/1971] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:15.194942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d42684-5911-4c87-93a5-6353232d2f7b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.195271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d42684-5911-4c87-93a5-6353232d2f7b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.195629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d42684-5911-4c87-93a5-6353232d2f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.223134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-f3d42684-5911-4c87-93a5-6353232d2f7b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:15.225089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f3d42684-5911-4c87-93a5-6353232d2f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:15.244454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 988/1972] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:15.244454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d42684-5911-4c87-93a5-6353232d2f7b None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:15.256802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d42684-5911-4c87-93a5-6353232d2f7b None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:15.264179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9077a-32b5-41a7-9a38-907fcd620ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:15.264179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9077a-32b5-41a7-9a38-907fcd620ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.264179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9077a-32b5-41a7-9a38-907fcd620ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.264179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9077a-32b5-41a7-9a38-907fcd620ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.265349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9077a-32b5-41a7-9a38-907fcd620ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:15.266934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9077a-32b5-41a7-9a38-907fcd620ac7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:15.286631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 989/1973] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:15.287519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9077a-32b5-41a7-9a38-907fcd620ac7 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:15.316373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 985/1974] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:15.319309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8edd2-de3a-4181-8c65-798e42a2da7d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:15.326461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9077a-32b5-41a7-9a38-907fcd620ac7 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:15.347313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358232c-521b-4c44-b414-db44df4983de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:15.347909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358232c-521b-4c44-b414-db44df4983de admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.351578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358232c-521b-4c44-b414-db44df4983de admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.352111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358232c-521b-4c44-b414-db44df4983de admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.353211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358232c-521b-4c44-b414-db44df4983de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:15.354332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358232c-521b-4c44-b414-db44df4983de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:15.357122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358232c-521b-4c44-b414-db44df4983de admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:15.360793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8edd2-de3a-4181-8c65-798e42a2da7d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:15.366631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab41d0b-453a-44e1-b9e2-56ea649c0180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:15.367437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41d0b-453a-44e1-b9e2-56ea649c0180 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.367839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41d0b-453a-44e1-b9e2-56ea649c0180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.368324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41d0b-453a-44e1-b9e2-56ea649c0180 admin admin] PATH_INFO: `/v3/projects/29d9df82bff0499ea1f97b0f5170cf4c/users/7dc2580956b1484d836aba9ebc32d8b3/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.383185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 990/1975] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:15.387465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab41d0b-453a-44e1-b9e2-56ea649c0180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29d9df82bff0499ea1f97b0f5170cf4c, user_id=7dc2580956b1484d836aba9ebc32d8b3, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:15.387746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358232c-521b-4c44-b414-db44df4983de admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:15.389928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1358232c-521b-4c44-b414-db44df4983de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:15.393105 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab41d0b-453a-44e1-b9e2-56ea649c0180 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:15.492177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358232c-521b-4c44-b414-db44df4983de admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:15.496826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f7ce3-9470-4786-bc69-31a70e2a2b17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:15.497562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f7ce3-9470-4786-bc69-31a70e2a2b17 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.498495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f7ce3-9470-4786-bc69-31a70e2a2b17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.498495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f7ce3-9470-4786-bc69-31a70e2a2b17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.510406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f7ce3-9470-4786-bc69-31a70e2a2b17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:15.511905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f7ce3-9470-4786-bc69-31a70e2a2b17 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:15.523367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 991/1976] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:15 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:15.524045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f7ce3-9470-4786-bc69-31a70e2a2b17 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:15.542468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 986/1977] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:15 2022] PUT /identity/v3/projects/29d9df82bff0499ea1f97b0f5170cf4c/users/7dc2580956b1484d836aba9ebc32d8b3/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:15.553710 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f7ce3-9470-4786-bc69-31a70e2a2b17 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:15.555692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a4642-9c3d-4401-ad6f-4300ad9a8258 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.556147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a4642-9c3d-4401-ad6f-4300ad9a8258 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.556550 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a4642-9c3d-4401-ad6f-4300ad9a8258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.621039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7f7a4642-9c3d-4401-ad6f-4300ad9a8258 None None] MFA Rules not processed for user `7dc2580956b1484d836aba9ebc32d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:15.622334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7f7a4642-9c3d-4401-ad6f-4300ad9a8258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:15.649628 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94793781-9548-4ef8-a554-4f8e857d03c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:15.653421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94793781-9548-4ef8-a554-4f8e857d03c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.653421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94793781-9548-4ef8-a554-4f8e857d03c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.653421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94793781-9548-4ef8-a554-4f8e857d03c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.653421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94793781-9548-4ef8-a554-4f8e857d03c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:15.653791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94793781-9548-4ef8-a554-4f8e857d03c3 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:15.700121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 987/1978] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:15.701122 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a4642-9c3d-4401-ad6f-4300ad9a8258 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:15.750843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 992/1979] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:15 2022] POST /identity/v3/projects => generated 389 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:15.750843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94793781-9548-4ef8-a554-4f8e857d03c3 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:15.787680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a4642-9c3d-4401-ad6f-4300ad9a8258 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:15.829829 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94793781-9548-4ef8-a554-4f8e857d03c3 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:15.844611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82d5ed-5179-40ba-8f9c-c644e68c69a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:15.845245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82d5ed-5179-40ba-8f9c-c644e68c69a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.845522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82d5ed-5179-40ba-8f9c-c644e68c69a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.845796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82d5ed-5179-40ba-8f9c-c644e68c69a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.846598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82d5ed-5179-40ba-8f9c-c644e68c69a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:15.853411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82d5ed-5179-40ba-8f9c-c644e68c69a2 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:15.872382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37f8f7-51c4-4d18-88b1-ca3361cd7f53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:15.873193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37f8f7-51c4-4d18-88b1-ca3361cd7f53 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:15.879999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37f8f7-51c4-4d18-88b1-ca3361cd7f53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:15.879999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37f8f7-51c4-4d18-88b1-ca3361cd7f53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:15.879999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4c37f8f7-51c4-4d18-88b1-ca3361cd7f53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:15.981645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37f8f7-51c4-4d18-88b1-ca3361cd7f53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:15.983282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37f8f7-51c4-4d18-88b1-ca3361cd7f53 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:15.999095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 988/1980] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:16.006156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37f8f7-51c4-4d18-88b1-ca3361cd7f53 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:16.007852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 993/1981] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:15 2022] POST /identity/v3/projects => generated 385 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:16.009483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82d5ed-5179-40ba-8f9c-c644e68c69a2 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:16.014068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37f8f7-51c4-4d18-88b1-ca3361cd7f53 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:16.108774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fb706-e097-448e-b1c6-f206ed27d051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:16.109504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fb706-e097-448e-b1c6-f206ed27d051 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.109911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fb706-e097-448e-b1c6-f206ed27d051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.110282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fb706-e097-448e-b1c6-f206ed27d051 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.111247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fb706-e097-448e-b1c6-f206ed27d051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:16.113482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fb706-e097-448e-b1c6-f206ed27d051 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:16.157997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82d5ed-5179-40ba-8f9c-c644e68c69a2 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:16.175091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e25a9c-751b-4753-be4f-5b8f2479a2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:16.175986 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e25a9c-751b-4753-be4f-5b8f2479a2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.176414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e25a9c-751b-4753-be4f-5b8f2479a2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.176998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e25a9c-751b-4753-be4f-5b8f2479a2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.178436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e25a9c-751b-4753-be4f-5b8f2479a2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:16.182353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e25a9c-751b-4753-be4f-5b8f2479a2c7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:16.299410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 994/1982] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:16 2022] POST /identity/v3/users => generated 299 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:16.299690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 989/1983] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:15 2022] POST /identity/v3/users => generated 295 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:16.303566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e25a9c-751b-4753-be4f-5b8f2479a2c7 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:16.304872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fb706-e097-448e-b1c6-f206ed27d051 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:16.329537 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e25a9c-751b-4753-be4f-5b8f2479a2c7 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:16.331382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fb706-e097-448e-b1c6-f206ed27d051 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:16.347729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27247c4e-7095-402e-ab76-c18c014c6767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:16.348398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247c4e-7095-402e-ab76-c18c014c6767 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.348682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247c4e-7095-402e-ab76-c18c014c6767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.348960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247c4e-7095-402e-ab76-c18c014c6767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.349619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27247c4e-7095-402e-ab76-c18c014c6767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:16.351506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27247c4e-7095-402e-ab76-c18c014c6767 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:16.352041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413e3d3-8bfe-4818-9783-4d6d6696c062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:16.352795 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413e3d3-8bfe-4818-9783-4d6d6696c062 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.354124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413e3d3-8bfe-4818-9783-4d6d6696c062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.354124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413e3d3-8bfe-4818-9783-4d6d6696c062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.358083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413e3d3-8bfe-4818-9783-4d6d6696c062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:16.358083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413e3d3-8bfe-4818-9783-4d6d6696c062 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:16.373325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 990/1984] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:16.380334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 995/1985] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:16.387120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413e3d3-8bfe-4818-9783-4d6d6696c062 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:16.390914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27247c4e-7095-402e-ab76-c18c014c6767 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:16.406351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413e3d3-8bfe-4818-9783-4d6d6696c062 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:16.411341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27247c4e-7095-402e-ab76-c18c014c6767 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:16.412224 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652cba9-5225-492f-a446-bb2305a26b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:16.413130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652cba9-5225-492f-a446-bb2305a26b0e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.413504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652cba9-5225-492f-a446-bb2305a26b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.413985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652cba9-5225-492f-a446-bb2305a26b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.414730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652cba9-5225-492f-a446-bb2305a26b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:16.416747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652cba9-5225-492f-a446-bb2305a26b0e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:16.423684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c8439-4f01-40b6-9afa-b52f7172c4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:16.423990 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c8439-4f01-40b6-9afa-b52f7172c4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.424195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c8439-4f01-40b6-9afa-b52f7172c4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.424561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c8439-4f01-40b6-9afa-b52f7172c4c4 admin admin] PATH_INFO: `/v3/projects/8ed42bf5ceb940bfac1825895e928c97/users/95279f0e4ae140d798fca33b6f65cb28/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.438240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 991/1986] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:16.439896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652cba9-5225-492f-a446-bb2305a26b0e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:16.455640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652cba9-5225-492f-a446-bb2305a26b0e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:16.455841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c8439-4f01-40b6-9afa-b52f7172c4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed42bf5ceb940bfac1825895e928c97, user_id=95279f0e4ae140d798fca33b6f65cb28, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:16.459244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c8439-4f01-40b6-9afa-b52f7172c4c4 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:16.463315 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149655fd-f2a5-435d-8f9a-3e83672e385d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:16.464751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149655fd-f2a5-435d-8f9a-3e83672e385d admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.469344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149655fd-f2a5-435d-8f9a-3e83672e385d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.469344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149655fd-f2a5-435d-8f9a-3e83672e385d admin admin] PATH_INFO: `/v3/projects/42fbdbd2b9af47f9a791719ea2762e26/users/4f3a1086157a4e4791e6817ac0ff8a46/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.487163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149655fd-f2a5-435d-8f9a-3e83672e385d admin admin] RBAC: Authorizing `identity:create_grant(project_id=42fbdbd2b9af47f9a791719ea2762e26, user_id=4f3a1086157a4e4791e6817ac0ff8a46, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:16.492907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149655fd-f2a5-435d-8f9a-3e83672e385d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:16.576725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 996/1987] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:16 2022] PUT /identity/v3/projects/8ed42bf5ceb940bfac1825895e928c97/users/95279f0e4ae140d798fca33b6f65cb28/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:16.578530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c8439-4f01-40b6-9afa-b52f7172c4c4 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:16.598839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:16.605633 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 992/1988] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:16 2022] PUT /identity/v3/projects/42fbdbd2b9af47f9a791719ea2762e26/users/4f3a1086157a4e4791e6817ac0ff8a46/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:16.625311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff9074-dd36-473e-be05-f3ca93ec5f0b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.625592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff9074-dd36-473e-be05-f3ca93ec5f0b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.625869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff9074-dd36-473e-be05-f3ca93ec5f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.680903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-94ff9074-dd36-473e-be05-f3ca93ec5f0b None None] MFA Rules not processed for user `4f3a1086157a4e4791e6817ac0ff8a46`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:16.682592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-94ff9074-dd36-473e-be05-f3ca93ec5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:16.725725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c8439-4f01-40b6-9afa-b52f7172c4c4 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:16.811743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 993/1989] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:16.815901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff9074-dd36-473e-be05-f3ca93ec5f0b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:16.829279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b3e85-8a83-482e-8e43-2ceded688b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:16.829931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b3e85-8a83-482e-8e43-2ceded688b8d admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.830257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b3e85-8a83-482e-8e43-2ceded688b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.830494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b3e85-8a83-482e-8e43-2ceded688b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.831465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b3e85-8a83-482e-8e43-2ceded688b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:16.833080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b3e85-8a83-482e-8e43-2ceded688b8d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:16.844580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 997/1990] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:16.849114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b3e85-8a83-482e-8e43-2ceded688b8d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:16.862736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b3e85-8a83-482e-8e43-2ceded688b8d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:16.868529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529ad10-5008-416f-b00a-1800e697bcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:16.869204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ad10-5008-416f-b00a-1800e697bcba admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.869479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ad10-5008-416f-b00a-1800e697bcba admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.869753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ad10-5008-416f-b00a-1800e697bcba admin admin] PATH_INFO: `/v3/projects/8ed42bf5ceb940bfac1825895e928c97/users/95279f0e4ae140d798fca33b6f65cb28/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.872323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529ad10-5008-416f-b00a-1800e697bcba admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed42bf5ceb940bfac1825895e928c97, user_id=95279f0e4ae140d798fca33b6f65cb28, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:16.875017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529ad10-5008-416f-b00a-1800e697bcba admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:16.899734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 998/1991] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:16 2022] PUT /identity/v3/projects/8ed42bf5ceb940bfac1825895e928c97/users/95279f0e4ae140d798fca33b6f65cb28/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:16.901970 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ca942-e641-4ea9-a82c-1cf37f132b61 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:16.902260 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ca942-e641-4ea9-a82c-1cf37f132b61 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:16.902673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ca942-e641-4ea9-a82c-1cf37f132b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:16.906684 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff9074-dd36-473e-be05-f3ca93ec5f0b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:16.946136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d14ca942-e641-4ea9-a82c-1cf37f132b61 None None] MFA Rules not processed for user `95279f0e4ae140d798fca33b6f65cb28`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:16.947793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d14ca942-e641-4ea9-a82c-1cf37f132b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:17.085003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c6d78-2810-4d31-a797-68cdafa28f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:17.088712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c6d78-2810-4d31-a797-68cdafa28f6f admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:17.088712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c6d78-2810-4d31-a797-68cdafa28f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:17.088712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c6d78-2810-4d31-a797-68cdafa28f6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:17.088712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c6d78-2810-4d31-a797-68cdafa28f6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:17.093116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c6d78-2810-4d31-a797-68cdafa28f6f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:17.128559 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 999/1992] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:17.141995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ca942-e641-4ea9-a82c-1cf37f132b61 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:17.231298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ca942-e641-4ea9-a82c-1cf37f132b61 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:17.242207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25126b-a497-4ee4-bdce-477896880609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:17.244218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25126b-a497-4ee4-bdce-477896880609 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:17.244890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25126b-a497-4ee4-bdce-477896880609 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:17.245638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25126b-a497-4ee4-bdce-477896880609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:17.383376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 994/1993] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:16 2022] POST /identity/v3/projects => generated 389 bytes in 568 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:17.398373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c6d78-2810-4d31-a797-68cdafa28f6f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:17.476052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25126b-a497-4ee4-bdce-477896880609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:17.477542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25126b-a497-4ee4-bdce-477896880609 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:17.506315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1000/1994] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:30:17.516802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c6d78-2810-4d31-a797-68cdafa28f6f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:17.524146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb923f-d7be-4ebc-ab2a-05512534b460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:17.528640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb923f-d7be-4ebc-ab2a-05512534b460 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:17.528640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb923f-d7be-4ebc-ab2a-05512534b460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:17.528640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb923f-d7be-4ebc-ab2a-05512534b460 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:17.528640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb923f-d7be-4ebc-ab2a-05512534b460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:17.529974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb923f-d7be-4ebc-ab2a-05512534b460 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:17.648571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 995/1995] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:17 2022] POST /identity/v3/users => generated 301 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:17.648668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25126b-a497-4ee4-bdce-477896880609 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:17.660851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25126b-a497-4ee4-bdce-477896880609 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:17.674432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148f806-32a1-4f38-a3a9-8dceaeaad767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:17.674432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148f806-32a1-4f38-a3a9-8dceaeaad767 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:17.674432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148f806-32a1-4f38-a3a9-8dceaeaad767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:17.674432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148f806-32a1-4f38-a3a9-8dceaeaad767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:17.674811 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148f806-32a1-4f38-a3a9-8dceaeaad767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:17.677574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148f806-32a1-4f38-a3a9-8dceaeaad767 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:17.711104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1001/1996] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:17.714665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb923f-d7be-4ebc-ab2a-05512534b460 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:17.741376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb923f-d7be-4ebc-ab2a-05512534b460 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:17.748847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:17.757252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37b1df-4284-4b0c-a7d7-b4604d565ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:17.758127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37b1df-4284-4b0c-a7d7-b4604d565ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:17.758412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37b1df-4284-4b0c-a7d7-b4604d565ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:17.758710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37b1df-4284-4b0c-a7d7-b4604d565ee0 admin admin] PATH_INFO: `/v3/projects/fffc78fe132f4c869cae323dedb29fd2/users/d67832f5c19a4b5f9adc3b2e82cf23d4/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:17.812525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37b1df-4284-4b0c-a7d7-b4604d565ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fffc78fe132f4c869cae323dedb29fd2, user_id=d67832f5c19a4b5f9adc3b2e82cf23d4, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:17.816565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37b1df-4284-4b0c-a7d7-b4604d565ee0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:17.923222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 996/1997] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:17 2022] PUT /identity/v3/projects/fffc78fe132f4c869cae323dedb29fd2/users/d67832f5c19a4b5f9adc3b2e82cf23d4/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:17.927554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148f806-32a1-4f38-a3a9-8dceaeaad767 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:17.939128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148f806-32a1-4f38-a3a9-8dceaeaad767 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:18.011810 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca869e-0430-42a4-a040-3176126d97fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:18.012515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca869e-0430-42a4-a040-3176126d97fc admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:18.012931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca869e-0430-42a4-a040-3176126d97fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:18.013251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca869e-0430-42a4-a040-3176126d97fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:18.013962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca869e-0430-42a4-a040-3176126d97fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:18.015702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca869e-0430-42a4-a040-3176126d97fc admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:18.026004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1002/1998] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:18.030350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37b1df-4284-4b0c-a7d7-b4604d565ee0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:18.041590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37b1df-4284-4b0c-a7d7-b4604d565ee0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:18.046791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6820b02-2046-4529-8139-661702af9902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:18.047606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6820b02-2046-4529-8139-661702af9902 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:18.047934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6820b02-2046-4529-8139-661702af9902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:18.048224 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6820b02-2046-4529-8139-661702af9902 admin admin] PATH_INFO: `/v3/projects/fffc78fe132f4c869cae323dedb29fd2/users/d67832f5c19a4b5f9adc3b2e82cf23d4/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:18.050156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6820b02-2046-4529-8139-661702af9902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fffc78fe132f4c869cae323dedb29fd2, user_id=d67832f5c19a4b5f9adc3b2e82cf23d4, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:18.053467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6820b02-2046-4529-8139-661702af9902 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:18.068452 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 997/1999] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:18 2022] PUT /identity/v3/projects/fffc78fe132f4c869cae323dedb29fd2/users/d67832f5c19a4b5f9adc3b2e82cf23d4/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:18.082296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9f8e2-c588-4e24-a95a-721c011b39ee None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:18.082867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9f8e2-c588-4e24-a95a-721c011b39ee None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:18.083368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9f8e2-c588-4e24-a95a-721c011b39ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:18.112714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-77d9f8e2-c588-4e24-a95a-721c011b39ee None None] MFA Rules not processed for user `d67832f5c19a4b5f9adc3b2e82cf23d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:18.114455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-77d9f8e2-c588-4e24-a95a-721c011b39ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:18.215311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1003/2000] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:18.227110 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6820b02-2046-4529-8139-661702af9902 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:18.228835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d6820b02-2046-4529-8139-661702af9902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:18.296265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6820b02-2046-4529-8139-661702af9902 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:18.300076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37012c4-f0bc-476f-b842-a1c7db35562f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:18.300682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37012c4-f0bc-476f-b842-a1c7db35562f service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:18.300960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37012c4-f0bc-476f-b842-a1c7db35562f service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:18.301229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37012c4-f0bc-476f-b842-a1c7db35562f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:18.317111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37012c4-f0bc-476f-b842-a1c7db35562f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:18.317111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37012c4-f0bc-476f-b842-a1c7db35562f service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:18.325897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 998/2001] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:18 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:18.522490 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9f8e2-c588-4e24-a95a-721c011b39ee None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:18.589575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9f8e2-c588-4e24-a95a-721c011b39ee None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:18.594256 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:18.594888 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:18.595165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:18.595435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] PATH_INFO: `/v3/users/4f3a1086157a4e4791e6817ac0ff8a46` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:18.596589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f3a1086157a4e4791e6817ac0ff8a46)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:18.599039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:18.814122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f3a1086157a4e4791e6817ac0ff8a46'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:18.814851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f3a1086157a4e4791e6817ac0ff8a46'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:18.914735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f3a1086157a4e4791e6817ac0ff8a46'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:18.930212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f3a1086157a4e4791e6817ac0ff8a46'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:18.930920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f3a1086157a4e4791e6817ac0ff8a46'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:18.949047 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1004/2002] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:18 2022] DELETE /identity/v3/users/4f3a1086157a4e4791e6817ac0ff8a46 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:18.954032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54ca2a-bd55-440c-bfe6-b243b0f8787f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:18.954539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54ca2a-bd55-440c-bfe6-b243b0f8787f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:18.954699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54ca2a-bd55-440c-bfe6-b243b0f8787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:18.968572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:18.985378 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2c54ca2a-bd55-440c-bfe6-b243b0f8787f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:18.986913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2c54ca2a-bd55-440c-bfe6-b243b0f8787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:19.062818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 999/2003] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:19.075948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:19.077392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:19.143455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5f766-332d-41a3-88b2-bd4bc5b67465 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:19.147912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc41b0d-b1c7-4b94-bac2-a6fadea1c42e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:19.148644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc41b0d-b1c7-4b94-bac2-a6fadea1c42e service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:19.149024 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc41b0d-b1c7-4b94-bac2-a6fadea1c42e service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:19.149492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc41b0d-b1c7-4b94-bac2-a6fadea1c42e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:19.163470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc41b0d-b1c7-4b94-bac2-a6fadea1c42e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:19.165207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc41b0d-b1c7-4b94-bac2-a6fadea1c42e service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:19.175994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1005/2004] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:19.300932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:19.619696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54ca2a-bd55-440c-bfe6-b243b0f8787f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:19.621458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2c54ca2a-bd55-440c-bfe6-b243b0f8787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:19.635964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54ca2a-bd55-440c-bfe6-b243b0f8787f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:19.642573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:19.643831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:19.644703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:19.645472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] PATH_INFO: `/v3/projects/42fbdbd2b9af47f9a791719ea2762e26` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:19.647863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42fbdbd2b9af47f9a791719ea2762e26)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:19.650126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:19.812024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42fbdbd2b9af47f9a791719ea2762e26', 'request_id': 'req-8eec8be1-7c86-4e53-a337-4ab119af3d46'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:19.824029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42fbdbd2b9af47f9a791719ea2762e26', 'request_id': 'req-8eec8be1-7c86-4e53-a337-4ab119af3d46'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:19.882604 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42fbdbd2b9af47f9a791719ea2762e26', 'request_id': 'req-8eec8be1-7c86-4e53-a337-4ab119af3d46'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:19.896828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] The token cache is being invalidate because project 42fbdbd2b9af47f9a791719ea2762e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:19.897125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:19.897812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:19.899480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1000/2005] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:19 2022] DELETE /identity/v3/projects/42fbdbd2b9af47f9a791719ea2762e26 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:19.903924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc41b0d-b1c7-4b94-bac2-a6fadea1c42e service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:19.915391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc41b0d-b1c7-4b94-bac2-a6fadea1c42e service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:19.978434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:19.979250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:19.979617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:19.979904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] PATH_INFO: `/v3/users/d67832f5c19a4b5f9adc3b2e82cf23d4` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:19.981112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d67832f5c19a4b5f9adc3b2e82cf23d4)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:19.983241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:20.015945 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:20.018009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:20.039250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec8be1-7c86-4e53-a337-4ab119af3d46 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:20.044051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:20.044704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:20.044978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:20.045281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] PATH_INFO: `/v3/users/8125ec2ad31b45b19b7d831fc11c8d26` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:20.046509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8125ec2ad31b45b19b7d831fc11c8d26)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:20.048650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:20.134718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:20.297313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd67832f5c19a4b5f9adc3b2e82cf23d4'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:20.298348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd67832f5c19a4b5f9adc3b2e82cf23d4'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:20.330901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8125ec2ad31b45b19b7d831fc11c8d26'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:20.331734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8125ec2ad31b45b19b7d831fc11c8d26'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:20.406055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd67832f5c19a4b5f9adc3b2e82cf23d4'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:20.419488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd67832f5c19a4b5f9adc3b2e82cf23d4'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:20.420219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b1ac3925-c6c4-4b1f-8b03-db48f749b0ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd67832f5c19a4b5f9adc3b2e82cf23d4'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:20.432945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1006/2006] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:19 2022] DELETE /identity/v3/users/d67832f5c19a4b5f9adc3b2e82cf23d4 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:20.494287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8125ec2ad31b45b19b7d831fc11c8d26'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:20.508354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8125ec2ad31b45b19b7d831fc11c8d26'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:20.509347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8125ec2ad31b45b19b7d831fc11c8d26'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:20.534145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1001/2007] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:20 2022] DELETE /identity/v3/users/8125ec2ad31b45b19b7d831fc11c8d26 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:20.540554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deec5e4-f5a6-428d-9f82-5e26f2add014 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:20.540844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deec5e4-f5a6-428d-9f82-5e26f2add014 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:20.541121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deec5e4-f5a6-428d-9f82-5e26f2add014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:20.560113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7deec5e4-f5a6-428d-9f82-5e26f2add014 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:20.561352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7deec5e4-f5a6-428d-9f82-5e26f2add014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:20.633484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1007/2008] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:20.643550 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:20.644981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:20.703591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:20.707601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39edbf-e486-4cc7-bc42-30f7d95faf27 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:20.712045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f5b8b-d558-4b43-a134-84d4a268e78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:20.712789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f5b8b-d558-4b43-a134-84d4a268e78d service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:20.713166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f5b8b-d558-4b43-a134-84d4a268e78d service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:20.713504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f5b8b-d558-4b43-a134-84d4a268e78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:20.726651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f5b8b-d558-4b43-a134-84d4a268e78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:20.728180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f5b8b-d558-4b43-a134-84d4a268e78d service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:20.738179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1002/2009] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:30:20.965943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deec5e4-f5a6-428d-9f82-5e26f2add014 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:20.967415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7deec5e4-f5a6-428d-9f82-5e26f2add014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:20.987572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deec5e4-f5a6-428d-9f82-5e26f2add014 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:20.998342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:20.999258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:20.999666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.000037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] PATH_INFO: `/v3/projects/fffc78fe132f4c869cae323dedb29fd2` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.001466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fffc78fe132f4c869cae323dedb29fd2)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.006547 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:21.152880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fffc78fe132f4c869cae323dedb29fd2', 'request_id': 'req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.158577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fffc78fe132f4c869cae323dedb29fd2', 'request_id': 'req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.168272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:21.243023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fffc78fe132f4c869cae323dedb29fd2', 'request_id': 'req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.251019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f5b8b-d558-4b43-a134-84d4a268e78d service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:21.253464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ec3f5b8b-d558-4b43-a134-84d4a268e78d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:21.258192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] The token cache is being invalidate because project fffc78fe132f4c869cae323dedb29fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:21.258565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.259356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-deeb9077-ccb8-4dfd-bc80-a6ba3a0ff212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.261064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1008/2010] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:20 2022] DELETE /identity/v3/projects/fffc78fe132f4c869cae323dedb29fd2 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:21.267568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a9745-a530-4169-8dc7-a189cfefd986 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.268186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a9745-a530-4169-8dc7-a189cfefd986 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.268475 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a9745-a530-4169-8dc7-a189cfefd986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.282641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f5b8b-d558-4b43-a134-84d4a268e78d service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:21.289356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ae3a9745-a530-4169-8dc7-a189cfefd986 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:21.290576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ae3a9745-a530-4169-8dc7-a189cfefd986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:21.345079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:21.345727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.346008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.346279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] PATH_INFO: `/v3/projects/6f6fb474a92c4dc78d377e1adf1cc8cb` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.347447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6fb474a92c4dc78d377e1adf1cc8cb)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.351614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:21.357926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1009/2011] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:21.367173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a244-8623-4419-a25e-70949173cf23 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.370854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a244-8623-4419-a25e-70949173cf23 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.370854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a244-8623-4419-a25e-70949173cf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.390933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e4c8a244-8623-4419-a25e-70949173cf23 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:21.392856 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e4c8a244-8623-4419-a25e-70949173cf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:21.404452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1010/2012] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:21.405614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8a244-8623-4419-a25e-70949173cf23 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:21.415855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8a244-8623-4419-a25e-70949173cf23 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:21.422007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abab71-3752-4c13-a7fe-248c9e369b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:21.422192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abab71-3752-4c13-a7fe-248c9e369b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.422554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abab71-3752-4c13-a7fe-248c9e369b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.422901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abab71-3752-4c13-a7fe-248c9e369b34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.423880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abab71-3752-4c13-a7fe-248c9e369b34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:21.424257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abab71-3752-4c13-a7fe-248c9e369b34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.426279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abab71-3752-4c13-a7fe-248c9e369b34 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:21.456404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1011/2013] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:21.456404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abab71-3752-4c13-a7fe-248c9e369b34 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:21.463843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abab71-3752-4c13-a7fe-248c9e369b34 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:21.471385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927891d-dbf6-4540-97e2-20bb6fb0051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:21.472113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927891d-dbf6-4540-97e2-20bb6fb0051c admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.474139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927891d-dbf6-4540-97e2-20bb6fb0051c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.474139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927891d-dbf6-4540-97e2-20bb6fb0051c admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.474675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927891d-dbf6-4540-97e2-20bb6fb0051c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.477822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927891d-dbf6-4540-97e2-20bb6fb0051c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:21.512809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6fb474a92c4dc78d377e1adf1cc8cb', 'request_id': 'req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.524322 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6fb474a92c4dc78d377e1adf1cc8cb', 'request_id': 'req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.617263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6fb474a92c4dc78d377e1adf1cc8cb', 'request_id': 'req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.628793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1012/2014] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:21 2022] POST /identity/v3/projects => generated 365 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:21.629901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927891d-dbf6-4540-97e2-20bb6fb0051c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:21.636561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] The token cache is being invalidate because project 6f6fb474a92c4dc78d377e1adf1cc8cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:21.636935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.637823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.640397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1003/2015] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:21 2022] DELETE /identity/v3/projects/6f6fb474a92c4dc78d377e1adf1cc8cb => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:21.644621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:21.646360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:21.700997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927891d-dbf6-4540-97e2-20bb6fb0051c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:21.706222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62dcd1-ebc5-428f-9305-034297b3b1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:21.706873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62dcd1-ebc5-428f-9305-034297b3b1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.707158 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62dcd1-ebc5-428f-9305-034297b3b1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.707433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62dcd1-ebc5-428f-9305-034297b3b1c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.708605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62dcd1-ebc5-428f-9305-034297b3b1c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.710568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62dcd1-ebc5-428f-9305-034297b3b1c2 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:21.727516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5ef8a-c3c4-4932-be10-6c7a553cf04e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:21.732908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:21.733576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.733909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.734252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] PATH_INFO: `/v3/users/46177fa6653a4d98b6f01a2e4c5e4764` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.735398 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46177fa6653a4d98b6f01a2e4c5e4764)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.737460 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:21.866889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1013/2016] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:21 2022] POST /identity/v3/users => generated 283 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:21.875984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62dcd1-ebc5-428f-9305-034297b3b1c2 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:21.877891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8e62dcd1-ebc5-428f-9305-034297b3b1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:21.899921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62dcd1-ebc5-428f-9305-034297b3b1c2 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:21.899921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95698453-f000-4363-aa5f-340b405c071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:21.899921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698453-f000-4363-aa5f-340b405c071f admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.899921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698453-f000-4363-aa5f-340b405c071f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.900281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698453-f000-4363-aa5f-340b405c071f admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.900383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698453-f000-4363-aa5f-340b405c071f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.902012 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698453-f000-4363-aa5f-340b405c071f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:21.913229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1014/2017] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:21.917629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95698453-f000-4363-aa5f-340b405c071f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:21.934360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95698453-f000-4363-aa5f-340b405c071f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:21.939188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db63df9-28ea-4ac9-bebf-3b2b3c29692b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:21.939871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db63df9-28ea-4ac9-bebf-3b2b3c29692b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.940176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db63df9-28ea-4ac9-bebf-3b2b3c29692b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.940454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db63df9-28ea-4ac9-bebf-3b2b3c29692b admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.941123 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db63df9-28ea-4ac9-bebf-3b2b3c29692b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.942731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db63df9-28ea-4ac9-bebf-3b2b3c29692b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:21.953679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1015/2018] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:21.958544 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db63df9-28ea-4ac9-bebf-3b2b3c29692b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:21.973014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db63df9-28ea-4ac9-bebf-3b2b3c29692b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:21.978881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f14ef3-51a5-438e-91a9-553b868db26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:21.979655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f14ef3-51a5-438e-91a9-553b868db26b admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:21.979940 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f14ef3-51a5-438e-91a9-553b868db26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:21.980212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f14ef3-51a5-438e-91a9-553b868db26b admin admin] PATH_INFO: `/v3/projects/7a28c0e3105842fb91cd60b48140fcc9/users/606b156a29274d85ba31f7682c2f682d/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:21.993711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46177fa6653a4d98b6f01a2e4c5e4764'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.994588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46177fa6653a4d98b6f01a2e4c5e4764'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:21.994900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f14ef3-51a5-438e-91a9-553b868db26b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a28c0e3105842fb91cd60b48140fcc9, user_id=606b156a29274d85ba31f7682c2f682d, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:21.997588 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f14ef3-51a5-438e-91a9-553b868db26b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:22.165666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46177fa6653a4d98b6f01a2e4c5e4764'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:22.183085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46177fa6653a4d98b6f01a2e4c5e4764'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:22.184072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9ffdb9a6-0153-447e-bfd4-dc4d5d2c4697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46177fa6653a4d98b6f01a2e4c5e4764'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:22.203347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1004/2019] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:21 2022] DELETE /identity/v3/users/46177fa6653a4d98b6f01a2e4c5e4764 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:22.387722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1016/2020] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:21 2022] PUT /identity/v3/projects/7a28c0e3105842fb91cd60b48140fcc9/users/606b156a29274d85ba31f7682c2f682d/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:22.400005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162afbd0-f71a-4432-b84c-bed5169d6ce2 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:22.400710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162afbd0-f71a-4432-b84c-bed5169d6ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:22.400710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162afbd0-f71a-4432-b84c-bed5169d6ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:22.431575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-162afbd0-f71a-4432-b84c-bed5169d6ce2 None None] MFA Rules not processed for user `606b156a29274d85ba31f7682c2f682d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:22.432834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-162afbd0-f71a-4432-b84c-bed5169d6ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:22.512668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1005/2021] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:22.517200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f14ef3-51a5-438e-91a9-553b868db26b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:22.519654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-91f14ef3-51a5-438e-91a9-553b868db26b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:22.579573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:22.603419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f14ef3-51a5-438e-91a9-553b868db26b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:22.609163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f194e-3bba-498c-a8e2-8fde883a8e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:22.609875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f194e-3bba-498c-a8e2-8fde883a8e44 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:22.610346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f194e-3bba-498c-a8e2-8fde883a8e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:22.610611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f194e-3bba-498c-a8e2-8fde883a8e44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:22.611655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f194e-3bba-498c-a8e2-8fde883a8e44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:22.613789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f194e-3bba-498c-a8e2-8fde883a8e44 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:22.732010 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162afbd0-f71a-4432-b84c-bed5169d6ce2 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:22.742990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-162afbd0-f71a-4432-b84c-bed5169d6ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:22.743132 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1017/2022] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:22 2022] POST /identity/v3/projects => generated 363 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:22.746136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f194e-3bba-498c-a8e2-8fde883a8e44 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:22.782958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162afbd0-f71a-4432-b84c-bed5169d6ce2 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:22.829820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f194e-3bba-498c-a8e2-8fde883a8e44 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:22.834629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27991b11-d41a-49ee-a3fa-56d2255ed7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:22.835241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27991b11-d41a-49ee-a3fa-56d2255ed7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:22.835695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27991b11-d41a-49ee-a3fa-56d2255ed7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:22.835981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27991b11-d41a-49ee-a3fa-56d2255ed7e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:22.836609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27991b11-d41a-49ee-a3fa-56d2255ed7e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:22.838539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27991b11-d41a-49ee-a3fa-56d2255ed7e2 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:22.856888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:22.857618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:22.857960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:22.858300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] PATH_INFO: `/v3/projects/3568f2075b4d4e5981741ebd48c20fa9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:22.859585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3568f2075b4d4e5981741ebd48c20fa9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:22.861580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:22.978219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1018/2023] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:22 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:22.979041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27991b11-d41a-49ee-a3fa-56d2255ed7e2 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:22.980740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-27991b11-d41a-49ee-a3fa-56d2255ed7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:22.993365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27991b11-d41a-49ee-a3fa-56d2255ed7e2 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:22.998835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:22.999521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:22.999844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.000153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] PATH_INFO: `/v3/users/5ade72135cbf4cbda435722e759e3892` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.001318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ade72135cbf4cbda435722e759e3892)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:23.003467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:23.016242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3568f2075b4d4e5981741ebd48c20fa9', 'request_id': 'req-d1a33245-0684-443e-94f1-6c8d9fa77a65'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:23.021759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3568f2075b4d4e5981741ebd48c20fa9', 'request_id': 'req-d1a33245-0684-443e-94f1-6c8d9fa77a65'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:23.063216 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:23.093912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3568f2075b4d4e5981741ebd48c20fa9', 'request_id': 'req-d1a33245-0684-443e-94f1-6c8d9fa77a65'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:23.107178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] The token cache is being invalidate because project 3568f2075b4d4e5981741ebd48c20fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:23.107463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:23.108121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:23.109590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1006/2024] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:22 2022] DELETE /identity/v3/projects/3568f2075b4d4e5981741ebd48c20fa9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:23.111075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:23.113458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:23.182151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a33245-0684-443e-94f1-6c8d9fa77a65 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:23.187128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747032b-e195-48e7-a518-02c4a38d3645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:23.187773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747032b-e195-48e7-a518-02c4a38d3645 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.188053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747032b-e195-48e7-a518-02c4a38d3645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.188332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747032b-e195-48e7-a518-02c4a38d3645 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.188981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747032b-e195-48e7-a518-02c4a38d3645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:23.190587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747032b-e195-48e7-a518-02c4a38d3645 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:23.201268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1007/2025] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:23.202527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d37785-e559-4c82-b27b-1deaa8a11a00 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.202811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d37785-e559-4c82-b27b-1deaa8a11a00 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.203083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d37785-e559-4c82-b27b-1deaa8a11a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.227393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-19d37785-e559-4c82-b27b-1deaa8a11a00 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:23.228772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-19d37785-e559-4c82-b27b-1deaa8a11a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:23.232716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ade72135cbf4cbda435722e759e3892'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:23.233331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ade72135cbf4cbda435722e759e3892'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:23.237643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1008/2026] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:23.238475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d37785-e559-4c82-b27b-1deaa8a11a00 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:23.247982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d37785-e559-4c82-b27b-1deaa8a11a00 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:23.252883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e75ff-eebc-4edc-a7f2-f2756475e41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:23.253538 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e75ff-eebc-4edc-a7f2-f2756475e41f admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.253817 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e75ff-eebc-4edc-a7f2-f2756475e41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.254090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e75ff-eebc-4edc-a7f2-f2756475e41f admin admin] PATH_INFO: `/v3/projects/0d79c1cbff0b45a4a880b780081ce84f/users/591c3d86556a4680b7bcbd8fd7318f58/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.268633 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e75ff-eebc-4edc-a7f2-f2756475e41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d79c1cbff0b45a4a880b780081ce84f, user_id=591c3d86556a4680b7bcbd8fd7318f58, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:23.271891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e75ff-eebc-4edc-a7f2-f2756475e41f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:23.328765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ade72135cbf4cbda435722e759e3892'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:23.335565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1009/2027] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:23 2022] PUT /identity/v3/projects/0d79c1cbff0b45a4a880b780081ce84f/users/591c3d86556a4680b7bcbd8fd7318f58/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:23.336903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f398256-3691-41b5-8204-b3e3c32ef067 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.337186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f398256-3691-41b5-8204-b3e3c32ef067 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.337454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f398256-3691-41b5-8204-b3e3c32ef067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.343232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ade72135cbf4cbda435722e759e3892'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:23.344107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ade72135cbf4cbda435722e759e3892'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:23.358051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1f398256-3691-41b5-8204-b3e3c32ef067 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:23.359253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1f398256-3691-41b5-8204-b3e3c32ef067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:23.360296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1019/2028] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:22 2022] DELETE /identity/v3/users/5ade72135cbf4cbda435722e759e3892 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:23.361429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:23.362758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:23.427870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1010/2029] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:23.429274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c3fec-9ca8-4c70-b70a-b1a4b0343f14 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.429769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c3fec-9ca8-4c70-b70a-b1a4b0343f14 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.430046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c3fec-9ca8-4c70-b70a-b1a4b0343f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.432625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c39e4d-b67e-4e27-a6cc-a7f1ff26bd3d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:23.437217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae49ba-ef5f-4190-afec-30d44b77acbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:23.438015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae49ba-ef5f-4190-afec-30d44b77acbe admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.438298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae49ba-ef5f-4190-afec-30d44b77acbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.438601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae49ba-ef5f-4190-afec-30d44b77acbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.439206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae49ba-ef5f-4190-afec-30d44b77acbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:23.440864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae49ba-ef5f-4190-afec-30d44b77acbe admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:23.449418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-899c3fec-9ca8-4c70-b70a-b1a4b0343f14 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:23.450793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-899c3fec-9ca8-4c70-b70a-b1a4b0343f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:23.451958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1020/2030] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:23.453134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae49ba-ef5f-4190-afec-30d44b77acbe admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:23.460631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1011/2031] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:23.465658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae49ba-ef5f-4190-afec-30d44b77acbe admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:23.468954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c3fec-9ca8-4c70-b70a-b1a4b0343f14 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:23.470656 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d30fc-043b-4fd3-b48c-86b599bc4a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:23.471601 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d30fc-043b-4fd3-b48c-86b599bc4a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.472045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d30fc-043b-4fd3-b48c-86b599bc4a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.472491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d30fc-043b-4fd3-b48c-86b599bc4a66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.473963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d30fc-043b-4fd3-b48c-86b599bc4a66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:23.474414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d30fc-043b-4fd3-b48c-86b599bc4a66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:23.476476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d30fc-043b-4fd3-b48c-86b599bc4a66 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:23.483020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c3fec-9ca8-4c70-b70a-b1a4b0343f14 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:23.488464 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333370f-5ebc-47e5-bf75-dbc75ee00b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:23.489203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333370f-5ebc-47e5-bf75-dbc75ee00b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.489478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333370f-5ebc-47e5-bf75-dbc75ee00b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.489744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333370f-5ebc-47e5-bf75-dbc75ee00b26 admin admin] PATH_INFO: `/v3/projects/0d79c1cbff0b45a4a880b780081ce84f/users/591c3d86556a4680b7bcbd8fd7318f58/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.490254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1021/2032] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:23.491074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d30fc-043b-4fd3-b48c-86b599bc4a66 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:23.492074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333370f-5ebc-47e5-bf75-dbc75ee00b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d79c1cbff0b45a4a880b780081ce84f, user_id=591c3d86556a4680b7bcbd8fd7318f58, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:23.492812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1d6d30fc-043b-4fd3-b48c-86b599bc4a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:23.495102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333370f-5ebc-47e5-bf75-dbc75ee00b26 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:23.505821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1012/2033] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:23 2022] PUT /identity/v3/projects/0d79c1cbff0b45a4a880b780081ce84f/users/591c3d86556a4680b7bcbd8fd7318f58/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:23.506694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333370f-5ebc-47e5-bf75-dbc75ee00b26 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:23.508730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f333370f-5ebc-47e5-bf75-dbc75ee00b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:23.552181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d30fc-043b-4fd3-b48c-86b599bc4a66 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:23.569988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333370f-5ebc-47e5-bf75-dbc75ee00b26 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:23.573724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83900597-0abc-44d4-b588-74c7777b43ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:23.574494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83900597-0abc-44d4-b588-74c7777b43ec service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.574796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83900597-0abc-44d4-b588-74c7777b43ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.575070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83900597-0abc-44d4-b588-74c7777b43ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.576870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-83900597-0abc-44d4-b588-74c7777b43ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:23.602830 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc91de-83ff-412d-b78f-09afb97fa1ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:23.603573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc91de-83ff-412d-b78f-09afb97fa1ca service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.603858 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc91de-83ff-412d-b78f-09afb97fa1ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.604145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc91de-83ff-412d-b78f-09afb97fa1ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.646212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83900597-0abc-44d4-b588-74c7777b43ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:23.647674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83900597-0abc-44d4-b588-74c7777b43ec service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:23.657407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1013/2034] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:23.658029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83900597-0abc-44d4-b588-74c7777b43ec service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:23.669667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc91de-83ff-412d-b78f-09afb97fa1ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:23.671322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc91de-83ff-412d-b78f-09afb97fa1ca service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:23.682079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1022/2035] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:23.683336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35dc35-2a8f-49a6-8a57-c7674cc8684c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.683661 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35dc35-2a8f-49a6-8a57-c7674cc8684c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.683933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35dc35-2a8f-49a6-8a57-c7674cc8684c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.717462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8b35dc35-2a8f-49a6-8a57-c7674cc8684c None None] MFA Rules not processed for user `591c3d86556a4680b7bcbd8fd7318f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:23.718681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8b35dc35-2a8f-49a6-8a57-c7674cc8684c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:23.727894 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83900597-0abc-44d4-b588-74c7777b43ec service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:23.731313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:23.744249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dea8dc-9f55-4ef8-9194-77983b28c585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:23.744249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dea8dc-9f55-4ef8-9194-77983b28c585 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.744249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dea8dc-9f55-4ef8-9194-77983b28c585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.744249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dea8dc-9f55-4ef8-9194-77983b28c585 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.745064 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dea8dc-9f55-4ef8-9194-77983b28c585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:23.747200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dea8dc-9f55-4ef8-9194-77983b28c585 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:23.836743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1023/2036] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:23.854200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1014/2037] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:23 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:23.862182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35dc35-2a8f-49a6-8a57-c7674cc8684c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:23.874395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35dc35-2a8f-49a6-8a57-c7674cc8684c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:23.897822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dea8dc-9f55-4ef8-9194-77983b28c585 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:23.899654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-72dea8dc-9f55-4ef8-9194-77983b28c585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:23.959458 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abeeac-efeb-4c70-8ecb-614b86a579fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:23.960356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abeeac-efeb-4c70-8ecb-614b86a579fa admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:23.960642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abeeac-efeb-4c70-8ecb-614b86a579fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:23.960913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abeeac-efeb-4c70-8ecb-614b86a579fa admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:23.961718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abeeac-efeb-4c70-8ecb-614b86a579fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:23.964204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abeeac-efeb-4c70-8ecb-614b86a579fa admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:23.995928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dea8dc-9f55-4ef8-9194-77983b28c585 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:23.999084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39639b1-615a-4ea1-a654-c91b286a4384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:23.999740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39639b1-615a-4ea1-a654-c91b286a4384 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:24.000015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39639b1-615a-4ea1-a654-c91b286a4384 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:24.000720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39639b1-615a-4ea1-a654-c91b286a4384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:24.023020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:24.063251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1024/2038] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:23 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:24.068289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abeeac-efeb-4c70-8ecb-614b86a579fa admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:24.080473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abeeac-efeb-4c70-8ecb-614b86a579fa admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:24.087042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f9ae8-5ca3-4dca-a87f-32031e9b28f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:24.087698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f9ae8-5ca3-4dca-a87f-32031e9b28f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:24.088257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f9ae8-5ca3-4dca-a87f-32031e9b28f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:24.088257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f9ae8-5ca3-4dca-a87f-32031e9b28f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:24.088877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f9ae8-5ca3-4dca-a87f-32031e9b28f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:24.091125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f9ae8-5ca3-4dca-a87f-32031e9b28f1 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:24.108732 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1025/2039] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:24.110929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f9ae8-5ca3-4dca-a87f-32031e9b28f1 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:24.124777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39639b1-615a-4ea1-a654-c91b286a4384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:24.132380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39639b1-615a-4ea1-a654-c91b286a4384 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:24.142060 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f9ae8-5ca3-4dca-a87f-32031e9b28f1 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:24.143633 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1015/2040] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:30:24.147263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ff3ee-a8c9-48f1-97f8-4039a42098e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:24.147930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ff3ee-a8c9-48f1-97f8-4039a42098e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:24.148212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ff3ee-a8c9-48f1-97f8-4039a42098e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:24.148507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ff3ee-a8c9-48f1-97f8-4039a42098e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:24.151999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ff3ee-a8c9-48f1-97f8-4039a42098e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:24.151999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ff3ee-a8c9-48f1-97f8-4039a42098e7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:24.164214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1026/2041] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:24.168203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39639b1-615a-4ea1-a654-c91b286a4384 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:24.182798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39639b1-615a-4ea1-a654-c91b286a4384 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:24.188056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb181e-702d-4341-8198-bb26a3128fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:24.188750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb181e-702d-4341-8198-bb26a3128fef admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:24.189032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb181e-702d-4341-8198-bb26a3128fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:24.189299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb181e-702d-4341-8198-bb26a3128fef admin admin] PATH_INFO: `/v3/projects/53455294ed464787a602cd30df706f7c/users/ba446852f7c644ed9f55d99aa7edcd19/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:24.207171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb181e-702d-4341-8198-bb26a3128fef admin admin] RBAC: Authorizing `identity:create_grant(project_id=53455294ed464787a602cd30df706f7c, user_id=ba446852f7c644ed9f55d99aa7edcd19, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:24.209961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb181e-702d-4341-8198-bb26a3128fef admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:24.306333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ff3ee-a8c9-48f1-97f8-4039a42098e7 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:24.308735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-833ff3ee-a8c9-48f1-97f8-4039a42098e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:24.312263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1016/2042] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:24 2022] PUT /identity/v3/projects/53455294ed464787a602cd30df706f7c/users/ba446852f7c644ed9f55d99aa7edcd19/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:24.326963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5091-9846-4ab8-8c30-f1977df35789 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:24.327333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5091-9846-4ab8-8c30-f1977df35789 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:24.327714 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5091-9846-4ab8-8c30-f1977df35789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:24.333199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ff3ee-a8c9-48f1-97f8-4039a42098e7 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:24.384636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c6fd5091-9846-4ab8-8c30-f1977df35789 None None] MFA Rules not processed for user `ba446852f7c644ed9f55d99aa7edcd19`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:24.385882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c6fd5091-9846-4ab8-8c30-f1977df35789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:24.407151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:24.407879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:24.408156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:24.408427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] PATH_INFO: `/v3/projects/8c8ab2cb4bc8454f8c99dd03c5b6608e` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:24.409580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c8ab2cb4bc8454f8c99dd03c5b6608e)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:24.421051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:24.472249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1017/2043] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:24.473698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd5091-9846-4ab8-8c30-f1977df35789 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:24.539799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd5091-9846-4ab8-8c30-f1977df35789 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:24.543985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadc930-259c-43f6-ade4-41f2ad6bfd96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:24.544607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadc930-259c-43f6-ade4-41f2ad6bfd96 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:24.544885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadc930-259c-43f6-ade4-41f2ad6bfd96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:24.545276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadc930-259c-43f6-ade4-41f2ad6bfd96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:24.547106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-acadc930-259c-43f6-ade4-41f2ad6bfd96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:24.586763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c8ab2cb4bc8454f8c99dd03c5b6608e', 'request_id': 'req-d034003b-ebed-4c88-9237-868e6c0a857b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:24.603444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c8ab2cb4bc8454f8c99dd03c5b6608e', 'request_id': 'req-d034003b-ebed-4c88-9237-868e6c0a857b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:24.643995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadc930-259c-43f6-ade4-41f2ad6bfd96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:24.645881 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadc930-259c-43f6-ade4-41f2ad6bfd96 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:24.655738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1018/2044] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:24.656481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadc930-259c-43f6-ade4-41f2ad6bfd96 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:24.657768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-acadc930-259c-43f6-ade4-41f2ad6bfd96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:24.708480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c8ab2cb4bc8454f8c99dd03c5b6608e', 'request_id': 'req-d034003b-ebed-4c88-9237-868e6c0a857b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:24.746015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] The token cache is being invalidate because project 8c8ab2cb4bc8454f8c99dd03c5b6608e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:24.746015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:24.746015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:24.748003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1027/2045] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:24 2022] DELETE /identity/v3/projects/8c8ab2cb4bc8454f8c99dd03c5b6608e => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:24.752575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:24.754473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:24.770039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadc930-259c-43f6-ade4-41f2ad6bfd96 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:24.831133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa474c-1bf9-44a2-973e-d79c55e7d487 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '237bbeebe2a949aba444353c1d2ee04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:24.831802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa474c-1bf9-44a2-973e-d79c55e7d487 service glance] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:24.832155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa474c-1bf9-44a2-973e-d79c55e7d487 service glance] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:24.832487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa474c-1bf9-44a2-973e-d79c55e7d487 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:24.834754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-40fa474c-1bf9-44a2-973e-d79c55e7d487 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:24.841901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034003b-ebed-4c88-9237-868e6c0a857b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:24.847016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:24.847708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:24.847987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:24.848319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] PATH_INFO: `/v3/users/d4a51da848a4444393bcbdadbe9dcd94` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:24.850054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a51da848a4444393bcbdadbe9dcd94)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:24.852378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:24.898809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa474c-1bf9-44a2-973e-d79c55e7d487 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:24.900341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa474c-1bf9-44a2-973e-d79c55e7d487 service glance] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:24.910191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1019/2046] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:24.995086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:25.053734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a51da848a4444393bcbdadbe9dcd94'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:25.054431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a51da848a4444393bcbdadbe9dcd94'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:25.147478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:25.154631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a51da848a4444393bcbdadbe9dcd94'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:25.170428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a51da848a4444393bcbdadbe9dcd94'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:25.171504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bec0b1c2-2aef-4d55-992e-276b8f371831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a51da848a4444393bcbdadbe9dcd94'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:25.191122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1028/2047] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:24 2022] DELETE /identity/v3/users/d4a51da848a4444393bcbdadbe9dcd94 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:25.717332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa474c-1bf9-44a2-973e-d79c55e7d487 service glance] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:25.736244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa474c-1bf9-44a2-973e-d79c55e7d487 service glance] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:25.811166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:25.811845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:25.812425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:25.812796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] PATH_INFO: `/v3/projects/323fbde6748e4e559fef1fb27459fb8a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:25.813950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=323fbde6748e4e559fef1fb27459fb8a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:25.816098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:25.955918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '323fbde6748e4e559fef1fb27459fb8a', 'request_id': 'req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:25.964163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '323fbde6748e4e559fef1fb27459fb8a', 'request_id': 'req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:26.102764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '323fbde6748e4e559fef1fb27459fb8a', 'request_id': 'req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:26.117026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] The token cache is being invalidate because project 323fbde6748e4e559fef1fb27459fb8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:26.117617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:26.118669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-43bf5cef-2897-4ae9-9f84-8caaa30cf1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:26.121053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1020/2048] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:25 2022] DELETE /identity/v3/projects/323fbde6748e4e559fef1fb27459fb8a => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:26.127230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0fe1c-3b03-419a-944f-5ea7ff23aa25 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:26.131148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0fe1c-3b03-419a-944f-5ea7ff23aa25 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:26.131148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0fe1c-3b03-419a-944f-5ea7ff23aa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:26.159119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7de0fe1c-3b03-419a-944f-5ea7ff23aa25 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:26.161163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7de0fe1c-3b03-419a-944f-5ea7ff23aa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:26.192579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:26.233602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1029/2049] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:26.242202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bf9a6-24ba-45fe-9e53-db3c58ad9f6d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:26.242478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bf9a6-24ba-45fe-9e53-db3c58ad9f6d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:26.242751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bf9a6-24ba-45fe-9e53-db3c58ad9f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:26.264578 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-335bf9a6-24ba-45fe-9e53-db3c58ad9f6d None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:26.265796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-335bf9a6-24ba-45fe-9e53-db3c58ad9f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:26.274260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1021/2050] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:26.278576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0fe1c-3b03-419a-944f-5ea7ff23aa25 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:26.288492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0fe1c-3b03-419a-944f-5ea7ff23aa25 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:26.293570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31046a8c-8c0b-491c-9a14-3d6254d43518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:26.294169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31046a8c-8c0b-491c-9a14-3d6254d43518 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:26.294446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31046a8c-8c0b-491c-9a14-3d6254d43518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:26.294719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31046a8c-8c0b-491c-9a14-3d6254d43518 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:26.295483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31046a8c-8c0b-491c-9a14-3d6254d43518 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:26.295819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31046a8c-8c0b-491c-9a14-3d6254d43518 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:26.297413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31046a8c-8c0b-491c-9a14-3d6254d43518 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:26.310192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1030/2051] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:26.316387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bf9a6-24ba-45fe-9e53-db3c58ad9f6d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:26.330013 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bf9a6-24ba-45fe-9e53-db3c58ad9f6d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:26.335272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8693626-065e-4ec4-b488-7dd9ec7fb9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:26.335951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8693626-065e-4ec4-b488-7dd9ec7fb9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:26.336227 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8693626-065e-4ec4-b488-7dd9ec7fb9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:26.336501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8693626-065e-4ec4-b488-7dd9ec7fb9f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:26.337138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8693626-065e-4ec4-b488-7dd9ec7fb9f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:26.339059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8693626-065e-4ec4-b488-7dd9ec7fb9f3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:26.446147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1022/2052] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:26 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:26.447474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31046a8c-8c0b-491c-9a14-3d6254d43518 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:26.462157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31046a8c-8c0b-491c-9a14-3d6254d43518 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:26.538910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b65308-2b4e-45f9-a5ed-bbbde3915251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:26.539806 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b65308-2b4e-45f9-a5ed-bbbde3915251 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:26.540090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b65308-2b4e-45f9-a5ed-bbbde3915251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:26.540362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b65308-2b4e-45f9-a5ed-bbbde3915251 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:26.540994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b65308-2b4e-45f9-a5ed-bbbde3915251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:26.543144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b65308-2b4e-45f9-a5ed-bbbde3915251 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:26.631991 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1031/2053] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:26 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:26.636440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8693626-065e-4ec4-b488-7dd9ec7fb9f3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:26.652325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8693626-065e-4ec4-b488-7dd9ec7fb9f3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:26.657814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fbafe-67de-446c-a968-5990769d24c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:26.658445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fbafe-67de-446c-a968-5990769d24c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:26.658724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fbafe-67de-446c-a968-5990769d24c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:26.658997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fbafe-67de-446c-a968-5990769d24c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:26.659685 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fbafe-67de-446c-a968-5990769d24c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:26.663169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fbafe-67de-446c-a968-5990769d24c4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:26.673986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1023/2054] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:26.679424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b65308-2b4e-45f9-a5ed-bbbde3915251 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:26.689003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:26.692861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b65308-2b4e-45f9-a5ed-bbbde3915251 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:26.697770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323a3ca-2644-4068-8d26-39d4606da69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:26.698448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323a3ca-2644-4068-8d26-39d4606da69b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:26.698730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323a3ca-2644-4068-8d26-39d4606da69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:26.699030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323a3ca-2644-4068-8d26-39d4606da69b admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:26.699720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323a3ca-2644-4068-8d26-39d4606da69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:26.701369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323a3ca-2644-4068-8d26-39d4606da69b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:26.712494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1032/2055] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:26.717023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fbafe-67de-446c-a968-5990769d24c4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:26.728715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fbafe-67de-446c-a968-5990769d24c4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:26.734085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b78cc-c61f-4142-a5af-3a35faef7bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:26.734768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b78cc-c61f-4142-a5af-3a35faef7bea admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:26.735037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b78cc-c61f-4142-a5af-3a35faef7bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:26.735314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b78cc-c61f-4142-a5af-3a35faef7bea admin admin] PATH_INFO: `/v3/projects/d5ecc5cd1d854b17bbe9ee678ecacfe3/users/a669f6cbac5a4d84a9e71c27c01969ff/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:26.752028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b78cc-c61f-4142-a5af-3a35faef7bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ecc5cd1d854b17bbe9ee678ecacfe3, user_id=a669f6cbac5a4d84a9e71c27c01969ff, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:26.754667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b78cc-c61f-4142-a5af-3a35faef7bea admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:26.881277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1024/2056] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:26 2022] PUT /identity/v3/projects/d5ecc5cd1d854b17bbe9ee678ecacfe3/users/a669f6cbac5a4d84a9e71c27c01969ff/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:26.886540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d5f49-ccde-4fc8-bfcc-aace08a059d9 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:26.886540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d5f49-ccde-4fc8-bfcc-aace08a059d9 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:26.886907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d5f49-ccde-4fc8-bfcc-aace08a059d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:26.908691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6c3d5f49-ccde-4fc8-bfcc-aace08a059d9 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:26.910025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6c3d5f49-ccde-4fc8-bfcc-aace08a059d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:27.010008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1033/2057] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:27.023525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b78cc-c61f-4142-a5af-3a35faef7bea admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:27.025689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3a0b78cc-c61f-4142-a5af-3a35faef7bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:27.114823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b78cc-c61f-4142-a5af-3a35faef7bea admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:27.120463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acaf140-974f-4110-a2c9-4f9956fc28af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:27.122390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaf140-974f-4110-a2c9-4f9956fc28af service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:27.122390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaf140-974f-4110-a2c9-4f9956fc28af service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:27.122390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaf140-974f-4110-a2c9-4f9956fc28af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:27.134826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acaf140-974f-4110-a2c9-4f9956fc28af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:27.136698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acaf140-974f-4110-a2c9-4f9956fc28af service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:27.151669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1025/2058] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:28.012814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:28.454371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:29.564609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23112200-1943-46ff-8a3d-593754f97b49 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:29.565338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23112200-1943-46ff-8a3d-593754f97b49 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:29.565338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23112200-1943-46ff-8a3d-593754f97b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:29.599272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-23112200-1943-46ff-8a3d-593754f97b49 None None] MFA Rules not processed for user `a669f6cbac5a4d84a9e71c27c01969ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:29.602827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-23112200-1943-46ff-8a3d-593754f97b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:29.682131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1034/2059] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:29.695995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acaf140-974f-4110-a2c9-4f9956fc28af service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:29.709988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7acaf140-974f-4110-a2c9-4f9956fc28af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:29.776158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acaf140-974f-4110-a2c9-4f9956fc28af service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:29.780048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd31260-e749-42c3-b955-d03d9620545b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:29.780664 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd31260-e749-42c3-b955-d03d9620545b service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:29.780942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd31260-e749-42c3-b955-d03d9620545b service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:29.781267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd31260-e749-42c3-b955-d03d9620545b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:29.795369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd31260-e749-42c3-b955-d03d9620545b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:29.798585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd31260-e749-42c3-b955-d03d9620545b service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:29.816259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1026/2060] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:30.034885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:30.291658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23112200-1943-46ff-8a3d-593754f97b49 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:30.293457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-23112200-1943-46ff-8a3d-593754f97b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:30.311265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23112200-1943-46ff-8a3d-593754f97b49 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:30.318342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:30.319004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:30.319325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:30.319654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] PATH_INFO: `/v3/users/7dc2580956b1484d836aba9ebc32d8b3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:30.320809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc2580956b1484d836aba9ebc32d8b3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:30.322823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:30.405792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:30.598498 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc2580956b1484d836aba9ebc32d8b3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:30.604116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc2580956b1484d836aba9ebc32d8b3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:30.714655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc2580956b1484d836aba9ebc32d8b3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:30.741818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc2580956b1484d836aba9ebc32d8b3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:30.752425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc2580956b1484d836aba9ebc32d8b3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:30.772396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1035/2061] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:30 2022] DELETE /identity/v3/users/7dc2580956b1484d836aba9ebc32d8b3 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:30.784663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25302dc1-31d7-4b3b-9b84-157c32fdac91 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:30.786001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25302dc1-31d7-4b3b-9b84-157c32fdac91 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:30.786991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25302dc1-31d7-4b3b-9b84-157c32fdac91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:30.820032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-25302dc1-31d7-4b3b-9b84-157c32fdac91 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:30.821778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-25302dc1-31d7-4b3b-9b84-157c32fdac91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:30.907028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1027/2062] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:30.917449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:30.920556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:30.997187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbe47d-5a12-48fc-9392-484003770a19 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:31.004373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d16ec4-9277-467c-ad28-c738efb5ad9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:31.006279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d16ec4-9277-467c-ad28-c738efb5ad9e service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:31.006735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d16ec4-9277-467c-ad28-c738efb5ad9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:31.006735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d16ec4-9277-467c-ad28-c738efb5ad9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:31.016635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d16ec4-9277-467c-ad28-c738efb5ad9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:31.018329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d16ec4-9277-467c-ad28-c738efb5ad9e service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:31.028459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1036/2063] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:31.569030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25302dc1-31d7-4b3b-9b84-157c32fdac91 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:31.570167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-25302dc1-31d7-4b3b-9b84-157c32fdac91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:31.593864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25302dc1-31d7-4b3b-9b84-157c32fdac91 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:31.604700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:31.604700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:31.605169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:31.605169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] PATH_INFO: `/v3/projects/29d9df82bff0499ea1f97b0f5170cf4c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:31.606382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=29d9df82bff0499ea1f97b0f5170cf4c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:31.608503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:31.719822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29d9df82bff0499ea1f97b0f5170cf4c', 'request_id': 'req-1a14a1ad-a174-43f3-947e-127d40494a8f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:31.725353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29d9df82bff0499ea1f97b0f5170cf4c', 'request_id': 'req-1a14a1ad-a174-43f3-947e-127d40494a8f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:31.783100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:31.805517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29d9df82bff0499ea1f97b0f5170cf4c', 'request_id': 'req-1a14a1ad-a174-43f3-947e-127d40494a8f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:31.818957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] The token cache is being invalidate because project 29d9df82bff0499ea1f97b0f5170cf4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:31.819247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:31.819930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1a14a1ad-a174-43f3-947e-127d40494a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:31.821463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1028/2064] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:31 2022] DELETE /identity/v3/projects/29d9df82bff0499ea1f97b0f5170cf4c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:31.825643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d16ec4-9277-467c-ad28-c738efb5ad9e service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:31.838491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d16ec4-9277-467c-ad28-c738efb5ad9e service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:31.892611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8829818-e1a9-41aa-ae49-30052a3d97f9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:31.892897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8829818-e1a9-41aa-ae49-30052a3d97f9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:31.893314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8829818-e1a9-41aa-ae49-30052a3d97f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:31.908076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:31.908739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:31.909111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:31.909301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] PATH_INFO: `/v3/users/95279f0e4ae140d798fca33b6f65cb28` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:31.910504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95279f0e4ae140d798fca33b6f65cb28)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:31.913180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:31.914774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b8829818-e1a9-41aa-ae49-30052a3d97f9 None None] MFA Rules not processed for user `a669f6cbac5a4d84a9e71c27c01969ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:31.916073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b8829818-e1a9-41aa-ae49-30052a3d97f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:31.982990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1029/2065] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:32.028809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:32.121647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8829818-e1a9-41aa-ae49-30052a3d97f9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:32.122279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b8829818-e1a9-41aa-ae49-30052a3d97f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:32.138344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8829818-e1a9-41aa-ae49-30052a3d97f9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:32.143045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:32.143720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:32.143997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:32.144291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] PATH_INFO: `/v3/users/606b156a29274d85ba31f7682c2f682d` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:32.146062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=606b156a29274d85ba31f7682c2f682d)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:32.148071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:32.148933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95279f0e4ae140d798fca33b6f65cb28'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:32.149794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95279f0e4ae140d798fca33b6f65cb28'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:32.234393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95279f0e4ae140d798fca33b6f65cb28'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:32.248651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95279f0e4ae140d798fca33b6f65cb28'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:32.249383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7cbc9921-973f-48b6-8713-c9e1873af315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95279f0e4ae140d798fca33b6f65cb28'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:32.262983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1037/2066] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:31 2022] DELETE /identity/v3/users/95279f0e4ae140d798fca33b6f65cb28 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:32.435907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '606b156a29274d85ba31f7682c2f682d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:32.436698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '606b156a29274d85ba31f7682c2f682d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:32.544707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '606b156a29274d85ba31f7682c2f682d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:32.561990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '606b156a29274d85ba31f7682c2f682d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:32.562707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '606b156a29274d85ba31f7682c2f682d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:32.578721 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1030/2067] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:32 2022] DELETE /identity/v3/users/606b156a29274d85ba31f7682c2f682d => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:32.588708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77add1f5-747a-4b79-87ce-f8c01b302f81 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:32.588996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77add1f5-747a-4b79-87ce-f8c01b302f81 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:32.589336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77add1f5-747a-4b79-87ce-f8c01b302f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:32.613950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-77add1f5-747a-4b79-87ce-f8c01b302f81 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:32.615448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-77add1f5-747a-4b79-87ce-f8c01b302f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:32.710696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1038/2068] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:32.754422 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:32.758525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:32.864652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38778937-9cd1-4db7-90a4-8991db9d4d1d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:32.874433 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a186c-ca6d-41b6-9240-43b82a4dc2c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:32.875045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a186c-ca6d-41b6-9240-43b82a4dc2c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:32.875319 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a186c-ca6d-41b6-9240-43b82a4dc2c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:32.875890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a186c-ca6d-41b6-9240-43b82a4dc2c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:32.901177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a186c-ca6d-41b6-9240-43b82a4dc2c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:32.903760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a186c-ca6d-41b6-9240-43b82a4dc2c6 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:32.924411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1031/2069] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:32.957137 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77add1f5-747a-4b79-87ce-f8c01b302f81 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:32.960029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-77add1f5-747a-4b79-87ce-f8c01b302f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:32.979253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77add1f5-747a-4b79-87ce-f8c01b302f81 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:32.988631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:32.990222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:32.990704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:32.991202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] PATH_INFO: `/v3/projects/8ed42bf5ceb940bfac1825895e928c97` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:32.992643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed42bf5ceb940bfac1825895e928c97)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:32.995101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:33.176214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:33.193195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed42bf5ceb940bfac1825895e928c97', 'request_id': 'req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:33.198764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed42bf5ceb940bfac1825895e928c97', 'request_id': 'req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:33.326239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed42bf5ceb940bfac1825895e928c97', 'request_id': 'req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:33.342377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] The token cache is being invalidate because project 8ed42bf5ceb940bfac1825895e928c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:33.342377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:33.342825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4e7f5da2-7f2b-4a1e-abb1-39ba1fd0ef5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:33.344951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1039/2070] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:32 2022] DELETE /identity/v3/projects/8ed42bf5ceb940bfac1825895e928c97 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:33.353304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7ceb-5b0a-4127-84be-56d7f033a1e6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:33.353736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7ceb-5b0a-4127-84be-56d7f033a1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:33.354079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7ceb-5b0a-4127-84be-56d7f033a1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:33.387293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-208c7ceb-5b0a-4127-84be-56d7f033a1e6 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:33.389442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-208c7ceb-5b0a-4127-84be-56d7f033a1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:33.434857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:33.522762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1032/2071] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:33.533151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf72dd-6dbe-46ec-916c-49df7ed75eb5 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:33.533691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf72dd-6dbe-46ec-916c-49df7ed75eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:33.534103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf72dd-6dbe-46ec-916c-49df7ed75eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:33.542244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7ceb-5b0a-4127-84be-56d7f033a1e6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:33.548713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-208c7ceb-5b0a-4127-84be-56d7f033a1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:33.572154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7ceb-5b0a-4127-84be-56d7f033a1e6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:33.577798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-26bf72dd-6dbe-46ec-916c-49df7ed75eb5 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:33.581243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-26bf72dd-6dbe-46ec-916c-49df7ed75eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:33.584139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:33.584997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:33.585437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:33.585856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] PATH_INFO: `/v3/projects/7a28c0e3105842fb91cd60b48140fcc9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:33.588238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a28c0e3105842fb91cd60b48140fcc9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:33.590608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:33.605430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1040/2072] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:33.610025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf72dd-6dbe-46ec-916c-49df7ed75eb5 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:33.631192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf72dd-6dbe-46ec-916c-49df7ed75eb5 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:33.640826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c6328-91a7-45ba-be80-8036bffef18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:33.646585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c6328-91a7-45ba-be80-8036bffef18f admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:33.646585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c6328-91a7-45ba-be80-8036bffef18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:33.646585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c6328-91a7-45ba-be80-8036bffef18f admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:33.646585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c6328-91a7-45ba-be80-8036bffef18f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:33.646585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c6328-91a7-45ba-be80-8036bffef18f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:33.648084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c6328-91a7-45ba-be80-8036bffef18f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:33.672628 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1041/2073] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:33.677252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c6328-91a7-45ba-be80-8036bffef18f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:33.690779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c6328-91a7-45ba-be80-8036bffef18f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:33.701539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935f21d-d768-4fec-9bbd-382e5018883f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:33.702162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935f21d-d768-4fec-9bbd-382e5018883f admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:33.702450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935f21d-d768-4fec-9bbd-382e5018883f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:33.702720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935f21d-d768-4fec-9bbd-382e5018883f admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:33.703948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935f21d-d768-4fec-9bbd-382e5018883f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:33.705450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935f21d-d768-4fec-9bbd-382e5018883f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:33.753964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a28c0e3105842fb91cd60b48140fcc9', 'request_id': 'req-60f80d32-9763-46ed-946c-b62ee8117998'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:33.766404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a28c0e3105842fb91cd60b48140fcc9', 'request_id': 'req-60f80d32-9763-46ed-946c-b62ee8117998'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:33.875483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a28c0e3105842fb91cd60b48140fcc9', 'request_id': 'req-60f80d32-9763-46ed-946c-b62ee8117998'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:33.894593 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1042/2074] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:33 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:33.898168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935f21d-d768-4fec-9bbd-382e5018883f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:33.909544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] The token cache is being invalidate because project 7a28c0e3105842fb91cd60b48140fcc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:33.912878 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:33.917071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:33.920953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1033/2075] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:33 2022] DELETE /identity/v3/projects/7a28c0e3105842fb91cd60b48140fcc9 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:33.927445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:33.932310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:34.066689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935f21d-d768-4fec-9bbd-382e5018883f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:34.086524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641244-8920-472a-af1e-8bcd1fc516e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:34.087564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641244-8920-472a-af1e-8bcd1fc516e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:34.096548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641244-8920-472a-af1e-8bcd1fc516e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:34.096548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641244-8920-472a-af1e-8bcd1fc516e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:34.096548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9641244-8920-472a-af1e-8bcd1fc516e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:34.096548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9641244-8920-472a-af1e-8bcd1fc516e5 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:34.107666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f80d32-9763-46ed-946c-b62ee8117998 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:34.118392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:34.119386 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:34.119901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:34.120360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] PATH_INFO: `/v3/users/591c3d86556a4680b7bcbd8fd7318f58` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:34.122709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] RBAC: Authorizing `identity:delete_user(user_id=591c3d86556a4680b7bcbd8fd7318f58)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:34.127890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:34.296118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1043/2076] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:33 2022] POST /identity/v3/users => generated 292 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:34.304180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641244-8920-472a-af1e-8bcd1fc516e5 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:34.304180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d9641244-8920-472a-af1e-8bcd1fc516e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:34.320127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641244-8920-472a-af1e-8bcd1fc516e5 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:34.325759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:34.331608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d851b-f542-4393-8cc8-ed8fd77a2b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:34.335003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d851b-f542-4393-8cc8-ed8fd77a2b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:34.335003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d851b-f542-4393-8cc8-ed8fd77a2b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:34.335003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d851b-f542-4393-8cc8-ed8fd77a2b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:34.335003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d851b-f542-4393-8cc8-ed8fd77a2b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:34.336664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d851b-f542-4393-8cc8-ed8fd77a2b67 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:34.352738 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1044/2077] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:34.356765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d851b-f542-4393-8cc8-ed8fd77a2b67 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:34.367387 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d851b-f542-4393-8cc8-ed8fd77a2b67 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:34.372248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33050cb-0d81-4855-b0e0-429ae269e3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:34.373245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33050cb-0d81-4855-b0e0-429ae269e3de admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:34.373525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33050cb-0d81-4855-b0e0-429ae269e3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:34.373796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33050cb-0d81-4855-b0e0-429ae269e3de admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:34.374446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33050cb-0d81-4855-b0e0-429ae269e3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:34.376101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33050cb-0d81-4855-b0e0-429ae269e3de admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:34.390247 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1045/2078] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:34.394195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33050cb-0d81-4855-b0e0-429ae269e3de admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:34.407900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33050cb-0d81-4855-b0e0-429ae269e3de admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:34.413414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0591cf-4ca3-4f2d-a774-ea15b1ab1cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:34.414084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0591cf-4ca3-4f2d-a774-ea15b1ab1cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:34.414501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0591cf-4ca3-4f2d-a774-ea15b1ab1cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:34.414954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0591cf-4ca3-4f2d-a774-ea15b1ab1cc7 admin admin] PATH_INFO: `/v3/projects/1d1c7062fcf945ce9ef2b1a57d436fca/users/45e72a1cad344482ad5889e7845952ac/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:34.440526 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591c3d86556a4680b7bcbd8fd7318f58'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:34.441534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591c3d86556a4680b7bcbd8fd7318f58'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:34.459868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0591cf-4ca3-4f2d-a774-ea15b1ab1cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1c7062fcf945ce9ef2b1a57d436fca, user_id=45e72a1cad344482ad5889e7845952ac, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:34.463730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0591cf-4ca3-4f2d-a774-ea15b1ab1cc7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:34.540348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:34.570264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591c3d86556a4680b7bcbd8fd7318f58'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:34.588102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1046/2079] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:34 2022] PUT /identity/v3/projects/1d1c7062fcf945ce9ef2b1a57d436fca/users/45e72a1cad344482ad5889e7845952ac/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:34.595145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c694e11-ca0b-4383-85c6-12a365297e42 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:34.595505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c694e11-ca0b-4383-85c6-12a365297e42 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:34.595870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c694e11-ca0b-4383-85c6-12a365297e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:34.610359 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591c3d86556a4680b7bcbd8fd7318f58'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:34.610359 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591c3d86556a4680b7bcbd8fd7318f58'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:34.657195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1034/2080] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:33 2022] DELETE /identity/v3/users/591c3d86556a4680b7bcbd8fd7318f58 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:34.665978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-2c694e11-ca0b-4383-85c6-12a365297e42 None None] MFA Rules not processed for user `45e72a1cad344482ad5889e7845952ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:34.667992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2c694e11-ca0b-4383-85c6-12a365297e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:34.809893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1047/2081] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:34.818435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:34.821130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:34.937361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb2ad8-773c-45f8-a619-f4d0d7ca318f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:34.951060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af1a85-3507-4032-8087-5eaad9c18d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:34.952628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af1a85-3507-4032-8087-5eaad9c18d04 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:34.953792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af1a85-3507-4032-8087-5eaad9c18d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:34.954645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af1a85-3507-4032-8087-5eaad9c18d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:34.972834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af1a85-3507-4032-8087-5eaad9c18d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:34.975752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af1a85-3507-4032-8087-5eaad9c18d04 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:35.006449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1035/2082] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:35.054951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:35.181018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:35.424348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c694e11-ca0b-4383-85c6-12a365297e42 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:35.431345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2c694e11-ca0b-4383-85c6-12a365297e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:35.626130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c694e11-ca0b-4383-85c6-12a365297e42 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:35.636224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:35.637722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:35.638062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:35.638588 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] PATH_INFO: `/v3/projects/0d79c1cbff0b45a4a880b780081ce84f` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:35.640539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d79c1cbff0b45a4a880b780081ce84f)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:35.642636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:35.960955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d79c1cbff0b45a4a880b780081ce84f', 'request_id': 'req-07b059bf-59e1-49d0-8cce-86f615566aeb'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:35.970171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d79c1cbff0b45a4a880b780081ce84f', 'request_id': 'req-07b059bf-59e1-49d0-8cce-86f615566aeb'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:35.978070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:36.124392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d79c1cbff0b45a4a880b780081ce84f', 'request_id': 'req-07b059bf-59e1-49d0-8cce-86f615566aeb'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:36.152208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] The token cache is being invalidate because project 0d79c1cbff0b45a4a880b780081ce84f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:36.152838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:36.154389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-07b059bf-59e1-49d0-8cce-86f615566aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:36.162991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c8187-a645-4a54-9268-74b6017f1086 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:36.163649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c8187-a645-4a54-9268-74b6017f1086 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:36.164211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c8187-a645-4a54-9268-74b6017f1086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:36.167252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1048/2083] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:35 2022] DELETE /identity/v3/projects/0d79c1cbff0b45a4a880b780081ce84f => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:36.197509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-347c8187-a645-4a54-9268-74b6017f1086 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:36.204267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-347c8187-a645-4a54-9268-74b6017f1086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:36.364119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:36.387729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1036/2084] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:36.397976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae6709-7791-45c7-98f1-4010db668f97 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:36.398279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae6709-7791-45c7-98f1-4010db668f97 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:36.398583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae6709-7791-45c7-98f1-4010db668f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:36.443592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-baae6709-7791-45c7-98f1-4010db668f97 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:36.444888 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-baae6709-7791-45c7-98f1-4010db668f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:36.456472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1049/2085] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:36.472995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c8187-a645-4a54-9268-74b6017f1086 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:36.490986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c8187-a645-4a54-9268-74b6017f1086 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:36.497243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6614b-d646-4f90-94dc-0f1d7622d80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:36.498051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6614b-d646-4f90-94dc-0f1d7622d80c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:36.498334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6614b-d646-4f90-94dc-0f1d7622d80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:36.498677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6614b-d646-4f90-94dc-0f1d7622d80c admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:36.499287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6614b-d646-4f90-94dc-0f1d7622d80c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:36.500994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6614b-d646-4f90-94dc-0f1d7622d80c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:36.502838 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6614b-d646-4f90-94dc-0f1d7622d80c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:36.517028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1037/2086] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:36.522477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae6709-7791-45c7-98f1-4010db668f97 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:36.537448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae6709-7791-45c7-98f1-4010db668f97 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:36.543464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcbec7-8a5f-40c5-80db-0039482be4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:36.544281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcbec7-8a5f-40c5-80db-0039482be4ba admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:36.544581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcbec7-8a5f-40c5-80db-0039482be4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:36.544962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcbec7-8a5f-40c5-80db-0039482be4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:36.546350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fcbec7-8a5f-40c5-80db-0039482be4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:36.548875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fcbec7-8a5f-40c5-80db-0039482be4ba admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:36.699998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1050/2087] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:36 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:36.703969 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6614b-d646-4f90-94dc-0f1d7622d80c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:36.724131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6614b-d646-4f90-94dc-0f1d7622d80c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:36.834149 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f38d09-e654-4215-81ce-54b46db45a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:36.835324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f38d09-e654-4215-81ce-54b46db45a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:36.835619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f38d09-e654-4215-81ce-54b46db45a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:36.835895 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f38d09-e654-4215-81ce-54b46db45a14 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:36.836570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f38d09-e654-4215-81ce-54b46db45a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:36.840173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f38d09-e654-4215-81ce-54b46db45a14 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:36.972755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1038/2088] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:36 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:36.973233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcbec7-8a5f-40c5-80db-0039482be4ba admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:36.994342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcbec7-8a5f-40c5-80db-0039482be4ba admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:37.004646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85e2b9-ddc1-4115-8b16-588cc03664c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:37.007874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e2b9-ddc1-4115-8b16-588cc03664c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:37.007874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e2b9-ddc1-4115-8b16-588cc03664c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:37.007874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e2b9-ddc1-4115-8b16-588cc03664c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:37.007874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85e2b9-ddc1-4115-8b16-588cc03664c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:37.008412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85e2b9-ddc1-4115-8b16-588cc03664c0 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:37.025451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1051/2089] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:37.030717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f38d09-e654-4215-81ce-54b46db45a14 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:37.053490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f38d09-e654-4215-81ce-54b46db45a14 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:37.068453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e70ba2-6d4f-4d2c-a882-c967623569b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:37.069089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e70ba2-6d4f-4d2c-a882-c967623569b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:37.069370 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e70ba2-6d4f-4d2c-a882-c967623569b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:37.069642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e70ba2-6d4f-4d2c-a882-c967623569b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:37.070313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e70ba2-6d4f-4d2c-a882-c967623569b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:37.073110 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e70ba2-6d4f-4d2c-a882-c967623569b8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:37.095580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1039/2090] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:37.101709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85e2b9-ddc1-4115-8b16-588cc03664c0 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:37.125476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85e2b9-ddc1-4115-8b16-588cc03664c0 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:37.134399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774edbe-223b-4209-9059-82f2383e38f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:37.135057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774edbe-223b-4209-9059-82f2383e38f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:37.137267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774edbe-223b-4209-9059-82f2383e38f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:37.137267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774edbe-223b-4209-9059-82f2383e38f4 admin admin] PATH_INFO: `/v3/projects/bba7f5ef8a5641f3aa0d94327f64d8bd/users/ffae25e528e0416885265c53b996f628/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:37.170681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774edbe-223b-4209-9059-82f2383e38f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba7f5ef8a5641f3aa0d94327f64d8bd, user_id=ffae25e528e0416885265c53b996f628, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:37.180979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774edbe-223b-4209-9059-82f2383e38f4 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:37.306863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1052/2091] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:37 2022] PUT /identity/v3/projects/bba7f5ef8a5641f3aa0d94327f64d8bd/users/ffae25e528e0416885265c53b996f628/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:37.319494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd839d5-268f-4605-a00d-f15c4277e407 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:37.319860 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd839d5-268f-4605-a00d-f15c4277e407 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:37.320076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd839d5-268f-4605-a00d-f15c4277e407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:37.382314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1fd839d5-268f-4605-a00d-f15c4277e407 None None] MFA Rules not processed for user `ffae25e528e0416885265c53b996f628`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:37.383605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1fd839d5-268f-4605-a00d-f15c4277e407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:37.539717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1040/2092] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:37.541107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774edbe-223b-4209-9059-82f2383e38f4 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:37.690921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774edbe-223b-4209-9059-82f2383e38f4 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:37.700178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bb790-583c-4dc9-8d38-4854ec0671bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:37.701637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb790-583c-4dc9-8d38-4854ec0671bd admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:37.702306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb790-583c-4dc9-8d38-4854ec0671bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:37.702834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb790-583c-4dc9-8d38-4854ec0671bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:37.704274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bb790-583c-4dc9-8d38-4854ec0671bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:37.709126 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bb790-583c-4dc9-8d38-4854ec0671bd admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:37.716127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:37.840245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:38.045122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1053/2093] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:37 2022] POST /identity/v3/projects => generated 385 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:38.072091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd839d5-268f-4605-a00d-f15c4277e407 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:38.091448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd839d5-268f-4605-a00d-f15c4277e407 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:38.225864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fb30c-ebf5-4d19-8eae-a1c0e5795112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:38.225864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fb30c-ebf5-4d19-8eae-a1c0e5795112 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:38.225864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fb30c-ebf5-4d19-8eae-a1c0e5795112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:38.225864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fb30c-ebf5-4d19-8eae-a1c0e5795112 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:38.226532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fb30c-ebf5-4d19-8eae-a1c0e5795112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:38.228757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fb30c-ebf5-4d19-8eae-a1c0e5795112 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:38.404068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1041/2094] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:38 2022] POST /identity/v3/users => generated 299 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:38.405145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bb790-583c-4dc9-8d38-4854ec0671bd admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:38.428319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bb790-583c-4dc9-8d38-4854ec0671bd admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:38.437041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee9998-ea5e-4e30-a72f-e75924b4d1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:38.437771 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee9998-ea5e-4e30-a72f-e75924b4d1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:38.437969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee9998-ea5e-4e30-a72f-e75924b4d1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:38.438222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee9998-ea5e-4e30-a72f-e75924b4d1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:38.438841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee9998-ea5e-4e30-a72f-e75924b4d1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:38.441576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee9998-ea5e-4e30-a72f-e75924b4d1ca admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:38.454606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1054/2095] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:38.462390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fb30c-ebf5-4d19-8eae-a1c0e5795112 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:38.484869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fb30c-ebf5-4d19-8eae-a1c0e5795112 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:38.511958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd62e41-6d2a-4cbb-ad09-7e6db3cb9c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:38.512961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd62e41-6d2a-4cbb-ad09-7e6db3cb9c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:38.513486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd62e41-6d2a-4cbb-ad09-7e6db3cb9c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:38.514161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd62e41-6d2a-4cbb-ad09-7e6db3cb9c17 admin admin] PATH_INFO: `/v3/projects/25ea8684be784df4b4d3b9a00603fbb3/users/9bf5799826214fafb0970f943e98131b/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:38.553407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd62e41-6d2a-4cbb-ad09-7e6db3cb9c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ea8684be784df4b4d3b9a00603fbb3, user_id=9bf5799826214fafb0970f943e98131b, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:38.557269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd62e41-6d2a-4cbb-ad09-7e6db3cb9c17 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:38.742622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1042/2096] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:38 2022] PUT /identity/v3/projects/25ea8684be784df4b4d3b9a00603fbb3/users/9bf5799826214fafb0970f943e98131b/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:38.746526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee9998-ea5e-4e30-a72f-e75924b4d1ca admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:38.767822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee9998-ea5e-4e30-a72f-e75924b4d1ca admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:38.869960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e0a04-eb60-4d9c-970e-bfe94b5b14c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:38.870602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e0a04-eb60-4d9c-970e-bfe94b5b14c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:38.871057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e0a04-eb60-4d9c-970e-bfe94b5b14c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:38.871486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e0a04-eb60-4d9c-970e-bfe94b5b14c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:38.872311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e0a04-eb60-4d9c-970e-bfe94b5b14c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:38.875476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e0a04-eb60-4d9c-970e-bfe94b5b14c4 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:38.889825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1055/2097] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:38.895329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd62e41-6d2a-4cbb-ad09-7e6db3cb9c17 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:38.911811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd62e41-6d2a-4cbb-ad09-7e6db3cb9c17 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:38.918648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cded6d3-f475-4766-8d0f-5e4e4f0ae277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:38.919401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded6d3-f475-4766-8d0f-5e4e4f0ae277 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:38.921092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded6d3-f475-4766-8d0f-5e4e4f0ae277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:38.921745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded6d3-f475-4766-8d0f-5e4e4f0ae277 admin admin] PATH_INFO: `/v3/projects/25ea8684be784df4b4d3b9a00603fbb3/users/9bf5799826214fafb0970f943e98131b/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:38.927366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cded6d3-f475-4766-8d0f-5e4e4f0ae277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ea8684be784df4b4d3b9a00603fbb3, user_id=9bf5799826214fafb0970f943e98131b, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:38.934232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cded6d3-f475-4766-8d0f-5e4e4f0ae277 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:38.958931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1043/2098] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:38 2022] PUT /identity/v3/projects/25ea8684be784df4b4d3b9a00603fbb3/users/9bf5799826214fafb0970f943e98131b/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:38.970137 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacce2f6-7f10-4a5d-8610-9235a73951f2 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:38.970660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacce2f6-7f10-4a5d-8610-9235a73951f2 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:38.971098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacce2f6-7f10-4a5d-8610-9235a73951f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:39.019731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-aacce2f6-7f10-4a5d-8610-9235a73951f2 None None] MFA Rules not processed for user `9bf5799826214fafb0970f943e98131b`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:39.021697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-aacce2f6-7f10-4a5d-8610-9235a73951f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:39.104344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee014805-6055-4939-932a-ce15fd2a28b3 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:39.104884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee014805-6055-4939-932a-ce15fd2a28b3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:39.104884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee014805-6055-4939-932a-ce15fd2a28b3 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:39.106198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1044/2099] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:30:39.113022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5ba3-9501-48e5-af3b-15bf33cf56d7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:39.114043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5ba3-9501-48e5-af3b-15bf33cf56d7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:39.114043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5ba3-9501-48e5-af3b-15bf33cf56d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:39.148618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-769f5ba3-9501-48e5-af3b-15bf33cf56d7 None None] MFA Rules not processed for user `90d65372febb414d82843f9dfa56f738`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:39.148801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1056/2100] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:39.148889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-769f5ba3-9501-48e5-af3b-15bf33cf56d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:39.163376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacce2f6-7f10-4a5d-8610-9235a73951f2 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:39.164966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-aacce2f6-7f10-4a5d-8610-9235a73951f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:39.274332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1045/2101] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:30:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:39.289807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41432ed6-7992-48e4-8a9f-0e0b00c1f4df None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:39.289994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41432ed6-7992-48e4-8a9f-0e0b00c1f4df None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:39.290262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41432ed6-7992-48e4-8a9f-0e0b00c1f4df None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:39.290960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacce2f6-7f10-4a5d-8610-9235a73951f2 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:39.292514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1046/2102] 10.222.0.62 () {54 vars in 961 bytes} [Mon Sep 5 12:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:30:39.296580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610efe5-18a2-44e5-b76e-ba6e2a68e8c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:39.298073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610efe5-18a2-44e5-b76e-ba6e2a68e8c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:39.299022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610efe5-18a2-44e5-b76e-ba6e2a68e8c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:39.299785 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610efe5-18a2-44e5-b76e-ba6e2a68e8c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:39.300650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4f570-38c4-473a-9b5f-270d3f168006 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:39.301195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4f570-38c4-473a-9b5f-270d3f168006 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:39.301692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4f570-38c4-473a-9b5f-270d3f168006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:39.313184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610efe5-18a2-44e5-b76e-ba6e2a68e8c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:39.315437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610efe5-18a2-44e5-b76e-ba6e2a68e8c5 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:39.326553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1057/2103] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:30:39.329945 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e2a4f570-38c4-473a-9b5f-270d3f168006 None None] MFA Rules not processed for user `90d65372febb414d82843f9dfa56f738`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:39.331357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e2a4f570-38c4-473a-9b5f-270d3f168006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:39.352881 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1047/2104] 10.222.0.62 () {58 vars in 1058 bytes} [Mon Sep 5 12:30:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:39.364198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610efe5-18a2-44e5-b76e-ba6e2a68e8c5 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:39.381309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4f570-38c4-473a-9b5f-270d3f168006 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:39.383427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e2a4f570-38c4-473a-9b5f-270d3f168006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:39.384253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610efe5-18a2-44e5-b76e-ba6e2a68e8c5 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:39.393812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff8226-9781-4ee1-977f-399703a01d85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:39.394703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff8226-9781-4ee1-977f-399703a01d85 service placement] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:39.394956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff8226-9781-4ee1-977f-399703a01d85 service placement] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:39.395369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff8226-9781-4ee1-977f-399703a01d85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:39.410715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff8226-9781-4ee1-977f-399703a01d85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:39.414071 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff8226-9781-4ee1-977f-399703a01d85 service placement] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:39.426780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1058/2105] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:39.445934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:39.529231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4f570-38c4-473a-9b5f-270d3f168006 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:39.546404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:39.547255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:39.547612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:39.547999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] PATH_INFO: `/v3/users/69f976789f894296a86b4cf651fdc3ae` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:39.550121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f976789f894296a86b4cf651fdc3ae)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:39.552001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:39.829566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f976789f894296a86b4cf651fdc3ae'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:39.830280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f976789f894296a86b4cf651fdc3ae'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:39.908261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f976789f894296a86b4cf651fdc3ae'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:39.929767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f976789f894296a86b4cf651fdc3ae'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:39.930587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-11478aac-66b1-4847-a1a6-fed2079bda8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f976789f894296a86b4cf651fdc3ae'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:39.945407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1048/2106] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:39 2022] DELETE /identity/v3/users/69f976789f894296a86b4cf651fdc3ae => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:40.328970 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff8226-9781-4ee1-977f-399703a01d85 service placement] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:40.348487 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff8226-9781-4ee1-977f-399703a01d85 service placement] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:40.432452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:40.433105 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:40.433388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:40.433663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] PATH_INFO: `/v3/projects/19911e0c0b444897b621ce06dbd8296a` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:40.434840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=19911e0c0b444897b621ce06dbd8296a)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:40.436872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:40.563869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19911e0c0b444897b621ce06dbd8296a', 'request_id': 'req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:40.566760 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:40.572559 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19911e0c0b444897b621ce06dbd8296a', 'request_id': 'req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:40.656894 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19911e0c0b444897b621ce06dbd8296a', 'request_id': 'req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:40.669341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] The token cache is being invalidate because project 19911e0c0b444897b621ce06dbd8296a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:40.669625 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:40.670328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-51d0ad36-ae5f-4fcb-b9ff-bac2b117af3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:40.672131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1059/2107] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:40 2022] DELETE /identity/v3/projects/19911e0c0b444897b621ce06dbd8296a => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:40.677492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9125ec8-db82-4eba-93ed-ab2c06b79b25 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:40.677686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9125ec8-db82-4eba-93ed-ab2c06b79b25 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:40.678032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9125ec8-db82-4eba-93ed-ab2c06b79b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:40.707885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d9125ec8-db82-4eba-93ed-ab2c06b79b25 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:40.709286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d9125ec8-db82-4eba-93ed-ab2c06b79b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:40.781936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1049/2108] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:40.790906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35008f5f-22c5-4848-a628-fe5202b7074d None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:40.791271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35008f5f-22c5-4848-a628-fe5202b7074d None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:40.791720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35008f5f-22c5-4848-a628-fe5202b7074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:40.802125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:40.815938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-35008f5f-22c5-4848-a628-fe5202b7074d None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:40.817183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-35008f5f-22c5-4848-a628-fe5202b7074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:40.826447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1060/2109] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:40.831157 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9125ec8-db82-4eba-93ed-ab2c06b79b25 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:40.844216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9125ec8-db82-4eba-93ed-ab2c06b79b25 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:40.850337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e352dc7-9b5e-4fe8-83d5-2c8152570e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:40.850977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e352dc7-9b5e-4fe8-83d5-2c8152570e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:40.851504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e352dc7-9b5e-4fe8-83d5-2c8152570e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:40.851930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e352dc7-9b5e-4fe8-83d5-2c8152570e04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:40.856376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e352dc7-9b5e-4fe8-83d5-2c8152570e04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:40.856376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e352dc7-9b5e-4fe8-83d5-2c8152570e04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:40.856376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e352dc7-9b5e-4fe8-83d5-2c8152570e04 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:40.869689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1050/2110] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:40.874384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35008f5f-22c5-4848-a628-fe5202b7074d None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:40.885238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35008f5f-22c5-4848-a628-fe5202b7074d None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:40.894554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6f8ba-104a-4443-8816-687046832843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:40.895445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6f8ba-104a-4443-8816-687046832843 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:40.895753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6f8ba-104a-4443-8816-687046832843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:40.896021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6f8ba-104a-4443-8816-687046832843 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:40.896671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6f8ba-104a-4443-8816-687046832843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:40.898659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6f8ba-104a-4443-8816-687046832843 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:40.951960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e352dc7-9b5e-4fe8-83d5-2c8152570e04 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:40.953834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3e352dc7-9b5e-4fe8-83d5-2c8152570e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:40.968300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e352dc7-9b5e-4fe8-83d5-2c8152570e04 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:40.973983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:40.974639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:40.974920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:40.975195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] PATH_INFO: `/v3/users/ffae25e528e0416885265c53b996f628` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:40.976919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffae25e528e0416885265c53b996f628)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:40.979049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:41.027956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1061/2111] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:40 2022] POST /identity/v3/projects => generated 381 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:41.032164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6f8ba-104a-4443-8816-687046832843 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:41.105909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6f8ba-104a-4443-8816-687046832843 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:41.110979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff6096-2718-4c5c-ab95-949be04dad55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:41.111693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff6096-2718-4c5c-ab95-949be04dad55 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:41.111973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff6096-2718-4c5c-ab95-949be04dad55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:41.112247 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff6096-2718-4c5c-ab95-949be04dad55 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:41.112897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff6096-2718-4c5c-ab95-949be04dad55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:41.115579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff6096-2718-4c5c-ab95-949be04dad55 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:41.312549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1062/2112] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:41 2022] POST /identity/v3/users => generated 291 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:41.316640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff6096-2718-4c5c-ab95-949be04dad55 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:41.321621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffae25e528e0416885265c53b996f628'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:41.322366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffae25e528e0416885265c53b996f628'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:41.328909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff6096-2718-4c5c-ab95-949be04dad55 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:41.334361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c083c90-1810-441e-b8be-3fa596111dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:41.335151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c083c90-1810-441e-b8be-3fa596111dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:41.335358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c083c90-1810-441e-b8be-3fa596111dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:41.335672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c083c90-1810-441e-b8be-3fa596111dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:41.336314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c083c90-1810-441e-b8be-3fa596111dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:41.338189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c083c90-1810-441e-b8be-3fa596111dbe admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:41.349194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1063/2113] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:41.353554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c083c90-1810-441e-b8be-3fa596111dbe admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:41.355301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1c083c90-1810-441e-b8be-3fa596111dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:41.369187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c083c90-1810-441e-b8be-3fa596111dbe admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:41.374025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac6218-2e29-4e68-a6f6-e33476766a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:41.375592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6218-2e29-4e68-a6f6-e33476766a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:41.375592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6218-2e29-4e68-a6f6-e33476766a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:41.375592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6218-2e29-4e68-a6f6-e33476766a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:41.375985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac6218-2e29-4e68-a6f6-e33476766a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:41.377507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac6218-2e29-4e68-a6f6-e33476766a67 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:41.388604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1064/2114] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:41.392700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac6218-2e29-4e68-a6f6-e33476766a67 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:41.403706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac6218-2e29-4e68-a6f6-e33476766a67 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:41.408510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffae25e528e0416885265c53b996f628'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:41.410476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc18cf-fbc7-48a2-99d5-a2b5b5b20f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:41.411812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc18cf-fbc7-48a2-99d5-a2b5b5b20f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:41.412087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc18cf-fbc7-48a2-99d5-a2b5b5b20f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:41.412358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc18cf-fbc7-48a2-99d5-a2b5b5b20f41 admin admin] PATH_INFO: `/v3/projects/6e204c0f072a40fb8d7802a755681c06/users/592d1e35510743fbb014e68f0e2189d5/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:41.427652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:41.429033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffae25e528e0416885265c53b996f628'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:41.430116 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8b40195d-57be-4932-8a3f-4897ffd714b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffae25e528e0416885265c53b996f628'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:41.432348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc18cf-fbc7-48a2-99d5-a2b5b5b20f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e204c0f072a40fb8d7802a755681c06, user_id=592d1e35510743fbb014e68f0e2189d5, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:41.437116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc18cf-fbc7-48a2-99d5-a2b5b5b20f41 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:41.446104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1051/2115] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:40 2022] DELETE /identity/v3/users/ffae25e528e0416885265c53b996f628 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:41.450531 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af757668-9ec2-4cfc-b995-20711b22b4aa None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:41.450816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af757668-9ec2-4cfc-b995-20711b22b4aa None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:41.455095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af757668-9ec2-4cfc-b995-20711b22b4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:41.474682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-af757668-9ec2-4cfc-b995-20711b22b4aa None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:41.475931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-af757668-9ec2-4cfc-b995-20711b22b4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:41.570008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1065/2116] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:41 2022] PUT /identity/v3/projects/6e204c0f072a40fb8d7802a755681c06/users/592d1e35510743fbb014e68f0e2189d5/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:41.580278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48f7db-e3c6-4abb-bb72-778374c55afd None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:41.580278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48f7db-e3c6-4abb-bb72-778374c55afd None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:41.580278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48f7db-e3c6-4abb-bb72-778374c55afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:41.580675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1052/2117] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:41.594539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af757668-9ec2-4cfc-b995-20711b22b4aa None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:41.596153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-af757668-9ec2-4cfc-b995-20711b22b4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:41.614842 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-0f48f7db-e3c6-4abb-bb72-778374c55afd None None] MFA Rules not processed for user `592d1e35510743fbb014e68f0e2189d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:41.616179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0f48f7db-e3c6-4abb-bb72-778374c55afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:41.669851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af757668-9ec2-4cfc-b995-20711b22b4aa None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:41.697172 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1066/2118] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:41.698480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48f7db-e3c6-4abb-bb72-778374c55afd None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:41.728297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7bf1f-e61a-48b8-a834-44cc5deb6df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:41.728921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7bf1f-e61a-48b8-a834-44cc5deb6df7 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:41.729202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7bf1f-e61a-48b8-a834-44cc5deb6df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:41.729480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7bf1f-e61a-48b8-a834-44cc5deb6df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:41.776368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48f7db-e3c6-4abb-bb72-778374c55afd None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:41.781790 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93134059-4312-4be6-90af-a1af65841f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:41.782400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93134059-4312-4be6-90af-a1af65841f95 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:41.782682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93134059-4312-4be6-90af-a1af65841f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:41.782957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93134059-4312-4be6-90af-a1af65841f95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:41.786558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93134059-4312-4be6-90af-a1af65841f95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:41.786558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93134059-4312-4be6-90af-a1af65841f95 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:41.796934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7bf1f-e61a-48b8-a834-44cc5deb6df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:41.799319 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7bf1f-e61a-48b8-a834-44cc5deb6df7 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:41.812328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1053/2119] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:30:41.837686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:41.893573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1067/2120] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:41 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:41.897680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7bf1f-e61a-48b8-a834-44cc5deb6df7 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:41.908095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7bf1f-e61a-48b8-a834-44cc5deb6df7 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:41.970284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e10de-eab2-4eeb-863e-9cfe5c8409cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:41.970929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e10de-eab2-4eeb-863e-9cfe5c8409cf admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:41.971343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e10de-eab2-4eeb-863e-9cfe5c8409cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:41.971671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e10de-eab2-4eeb-863e-9cfe5c8409cf admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:41.972293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e10de-eab2-4eeb-863e-9cfe5c8409cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:41.974794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e10de-eab2-4eeb-863e-9cfe5c8409cf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:42.110213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1054/2121] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:41 2022] POST /identity/v3/users => generated 296 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:42.113529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93134059-4312-4be6-90af-a1af65841f95 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:42.124549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93134059-4312-4be6-90af-a1af65841f95 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:42.130634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0541192-5c8e-4482-b4d0-ae5e2a4f1144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:42.131435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0541192-5c8e-4482-b4d0-ae5e2a4f1144 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:42.131751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0541192-5c8e-4482-b4d0-ae5e2a4f1144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:42.132087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0541192-5c8e-4482-b4d0-ae5e2a4f1144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:42.132736 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0541192-5c8e-4482-b4d0-ae5e2a4f1144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:42.136055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0541192-5c8e-4482-b4d0-ae5e2a4f1144 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:42.151740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1068/2122] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:42.153232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e10de-eab2-4eeb-863e-9cfe5c8409cf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:42.165710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e10de-eab2-4eeb-863e-9cfe5c8409cf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:42.170993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973c5e0-623f-416f-9f5d-ce77e26eeec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:42.171816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973c5e0-623f-416f-9f5d-ce77e26eeec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:42.172106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973c5e0-623f-416f-9f5d-ce77e26eeec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:42.172571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973c5e0-623f-416f-9f5d-ce77e26eeec3 admin admin] PATH_INFO: `/v3/projects/91952a63ccc047208d0734ef0e10c645/users/9c169b1feead4940857ddcdb5cbb8395/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:42.190394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973c5e0-623f-416f-9f5d-ce77e26eeec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91952a63ccc047208d0734ef0e10c645, user_id=9c169b1feead4940857ddcdb5cbb8395, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:42.194421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973c5e0-623f-416f-9f5d-ce77e26eeec3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:42.319089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1055/2123] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:42 2022] PUT /identity/v3/projects/91952a63ccc047208d0734ef0e10c645/users/9c169b1feead4940857ddcdb5cbb8395/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:42.326781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0541192-5c8e-4482-b4d0-ae5e2a4f1144 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:42.338400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0541192-5c8e-4482-b4d0-ae5e2a4f1144 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:42.348140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:42.410042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d5e17-255d-4167-90af-620f56661b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:42.410042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d5e17-255d-4167-90af-620f56661b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:42.410042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d5e17-255d-4167-90af-620f56661b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:42.410042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d5e17-255d-4167-90af-620f56661b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:42.410496 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d5e17-255d-4167-90af-620f56661b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:42.411834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d5e17-255d-4167-90af-620f56661b23 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:42.422848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1069/2124] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:42.428293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973c5e0-623f-416f-9f5d-ce77e26eeec3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:42.443025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973c5e0-623f-416f-9f5d-ce77e26eeec3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:42.448280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a29ed-ebb6-4477-bbde-683fa194f9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:42.451024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a29ed-ebb6-4477-bbde-683fa194f9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:42.452334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a29ed-ebb6-4477-bbde-683fa194f9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:42.452528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a29ed-ebb6-4477-bbde-683fa194f9ac admin admin] PATH_INFO: `/v3/projects/91952a63ccc047208d0734ef0e10c645/users/9c169b1feead4940857ddcdb5cbb8395/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:42.454663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a29ed-ebb6-4477-bbde-683fa194f9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=91952a63ccc047208d0734ef0e10c645, user_id=9c169b1feead4940857ddcdb5cbb8395, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:42.458748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a29ed-ebb6-4477-bbde-683fa194f9ac admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:42.472861 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1056/2125] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:42 2022] PUT /identity/v3/projects/91952a63ccc047208d0734ef0e10c645/users/9c169b1feead4940857ddcdb5cbb8395/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:42.489298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e072bf-23d0-4d66-87e0-312e8653906d None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:42.489588 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e072bf-23d0-4d66-87e0-312e8653906d None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:42.489865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e072bf-23d0-4d66-87e0-312e8653906d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:42.535733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-48e072bf-23d0-4d66-87e0-312e8653906d None None] MFA Rules not processed for user `9c169b1feead4940857ddcdb5cbb8395`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:42.537232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-48e072bf-23d0-4d66-87e0-312e8653906d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:42.628658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:42.647668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1070/2126] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:42.657255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a29ed-ebb6-4477-bbde-683fa194f9ac admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:42.659034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1f3a29ed-ebb6-4477-bbde-683fa194f9ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:42.756721 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a29ed-ebb6-4477-bbde-683fa194f9ac admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:42.761293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dd7c5-a6f9-430f-8bfa-403a13590278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:42.762128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dd7c5-a6f9-430f-8bfa-403a13590278 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:42.762475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dd7c5-a6f9-430f-8bfa-403a13590278 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:42.762875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dd7c5-a6f9-430f-8bfa-403a13590278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:42.775471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1dd7c5-a6f9-430f-8bfa-403a13590278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:42.776978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1dd7c5-a6f9-430f-8bfa-403a13590278 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:42.793638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1057/2127] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:42.811805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e072bf-23d0-4d66-87e0-312e8653906d None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:42.813996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-48e072bf-23d0-4d66-87e0-312e8653906d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:42.979434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:42.987763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e072bf-23d0-4d66-87e0-312e8653906d None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:43.007995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:43.008450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:43.008734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:43.009070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] PATH_INFO: `/v3/projects/bba7f5ef8a5641f3aa0d94327f64d8bd` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:43.010938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bba7f5ef8a5641f3aa0d94327f64d8bd)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:43.014698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:43.053418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:43.223029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bba7f5ef8a5641f3aa0d94327f64d8bd', 'request_id': 'req-b3489ba6-aa9e-41dd-a12b-13205f30ba77'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:43.232754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bba7f5ef8a5641f3aa0d94327f64d8bd', 'request_id': 'req-b3489ba6-aa9e-41dd-a12b-13205f30ba77'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:43.300593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dd7c5-a6f9-430f-8bfa-403a13590278 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:43.311270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dd7c5-a6f9-430f-8bfa-403a13590278 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:43.327488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bba7f5ef8a5641f3aa0d94327f64d8bd', 'request_id': 'req-b3489ba6-aa9e-41dd-a12b-13205f30ba77'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:43.346627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] The token cache is being invalidate because project bba7f5ef8a5641f3aa0d94327f64d8bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:43.347290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:43.347746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:43.350825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1071/2128] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:42 2022] DELETE /identity/v3/projects/bba7f5ef8a5641f3aa0d94327f64d8bd => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:43.359617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:43.362634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:43.399875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8fd37-9d13-47f1-b76d-a05274c2e1ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:43.399875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8fd37-9d13-47f1-b76d-a05274c2e1ff service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:43.399875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8fd37-9d13-47f1-b76d-a05274c2e1ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:43.399875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8fd37-9d13-47f1-b76d-a05274c2e1ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:43.400238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-14c8fd37-9d13-47f1-b76d-a05274c2e1ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:43.485492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8fd37-9d13-47f1-b76d-a05274c2e1ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:43.486952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8fd37-9d13-47f1-b76d-a05274c2e1ff service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:43.488239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3489ba6-aa9e-41dd-a12b-13205f30ba77 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:43.494796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:43.495438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:43.495752 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:43.496029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] PATH_INFO: `/v3/users/9bf5799826214fafb0970f943e98131b` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:43.496546 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1058/2129] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:30:43.498989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf5799826214fafb0970f943e98131b)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:43.501160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:43.548785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:43.738372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf5799826214fafb0970f943e98131b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:43.739031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf5799826214fafb0970f943e98131b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:43.823131 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf5799826214fafb0970f943e98131b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:43.834319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:43.848436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf5799826214fafb0970f943e98131b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:43.849159 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9fc8ad84-db59-4e8d-a6dc-82236a5af822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf5799826214fafb0970f943e98131b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:43.864148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1072/2130] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:43 2022] DELETE /identity/v3/users/9bf5799826214fafb0970f943e98131b => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:44.200175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:44.342120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8fd37-9d13-47f1-b76d-a05274c2e1ff service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:44.360560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8fd37-9d13-47f1-b76d-a05274c2e1ff service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:44.451178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:44.451898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:44.452178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:44.452531 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] PATH_INFO: `/v3/projects/25ea8684be784df4b4d3b9a00603fbb3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:44.453808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ea8684be784df4b4d3b9a00603fbb3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:44.457257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:44.591553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ea8684be784df4b4d3b9a00603fbb3', 'request_id': 'req-e6c9f010-f323-478b-9e0d-09b3601b8643'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:44.598162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ea8684be784df4b4d3b9a00603fbb3', 'request_id': 'req-e6c9f010-f323-478b-9e0d-09b3601b8643'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:44.685733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ea8684be784df4b4d3b9a00603fbb3', 'request_id': 'req-e6c9f010-f323-478b-9e0d-09b3601b8643'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:44.698813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] The token cache is being invalidate because project 25ea8684be784df4b4d3b9a00603fbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:44.698997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:44.699674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e6c9f010-f323-478b-9e0d-09b3601b8643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:44.701520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1059/2131] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:44 2022] DELETE /identity/v3/projects/25ea8684be784df4b4d3b9a00603fbb3 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:44.706574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18310a8-ce4c-4b44-a10c-c8848d9bae2f None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:44.707065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18310a8-ce4c-4b44-a10c-c8848d9bae2f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:44.707156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18310a8-ce4c-4b44-a10c-c8848d9bae2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:44.729480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-f18310a8-ce4c-4b44-a10c-c8848d9bae2f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:44.730929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f18310a8-ce4c-4b44-a10c-c8848d9bae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:44.732471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:44.803605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1073/2132] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:44.812504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5dd45-ced0-4493-bf87-ae4b2ec969ec None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:44.813027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5dd45-ced0-4493-bf87-ae4b2ec969ec None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:44.813305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5dd45-ced0-4493-bf87-ae4b2ec969ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:44.841456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1ba5dd45-ced0-4493-bf87-ae4b2ec969ec None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:44.842798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1ba5dd45-ced0-4493-bf87-ae4b2ec969ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:44.852916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1060/2133] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:44.858065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18310a8-ce4c-4b44-a10c-c8848d9bae2f None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:44.872488 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18310a8-ce4c-4b44-a10c-c8848d9bae2f None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:44.877440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b1a63-6a07-477b-b363-3a295aaa128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:44.878343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b1a63-6a07-477b-b363-3a295aaa128c admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:44.878633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b1a63-6a07-477b-b363-3a295aaa128c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:44.878910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b1a63-6a07-477b-b363-3a295aaa128c admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:44.880114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b1a63-6a07-477b-b363-3a295aaa128c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:44.880744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b1a63-6a07-477b-b363-3a295aaa128c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:44.882087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b1a63-6a07-477b-b363-3a295aaa128c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:44.895915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1074/2134] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:44.900183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5dd45-ced0-4493-bf87-ae4b2ec969ec None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:44.912183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5dd45-ced0-4493-bf87-ae4b2ec969ec None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:44.917863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c474761-9756-4d9b-93ef-fe91ee6ec60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:44.939814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c474761-9756-4d9b-93ef-fe91ee6ec60c admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:44.939814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c474761-9756-4d9b-93ef-fe91ee6ec60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:44.939814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c474761-9756-4d9b-93ef-fe91ee6ec60c admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:44.939814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c474761-9756-4d9b-93ef-fe91ee6ec60c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:44.939814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c474761-9756-4d9b-93ef-fe91ee6ec60c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:45.055627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1061/2135] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:44 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:45.061063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b1a63-6a07-477b-b363-3a295aaa128c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:45.076861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b1a63-6a07-477b-b363-3a295aaa128c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:45.125764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:45.147004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db63c2-6dc0-4970-b542-e8553d517d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:45.147827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db63c2-6dc0-4970-b542-e8553d517d43 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:45.148137 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db63c2-6dc0-4970-b542-e8553d517d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:45.148553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db63c2-6dc0-4970-b542-e8553d517d43 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:45.149232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db63c2-6dc0-4970-b542-e8553d517d43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:45.151724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db63c2-6dc0-4970-b542-e8553d517d43 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:45.249427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1075/2136] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:45 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:45.250125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c474761-9756-4d9b-93ef-fe91ee6ec60c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:45.266478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c474761-9756-4d9b-93ef-fe91ee6ec60c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:45.272527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ea4bf-ea0f-4434-b91c-84990cdafc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:45.273328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ea4bf-ea0f-4434-b91c-84990cdafc2c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:45.273607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ea4bf-ea0f-4434-b91c-84990cdafc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:45.273877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ea4bf-ea0f-4434-b91c-84990cdafc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:45.274527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ea4bf-ea0f-4434-b91c-84990cdafc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:45.276254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ea4bf-ea0f-4434-b91c-84990cdafc2c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:45.296164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1062/2137] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:45.297839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db63c2-6dc0-4970-b542-e8553d517d43 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:45.312545 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db63c2-6dc0-4970-b542-e8553d517d43 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:45.321490 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051abd33-4143-4e0b-934b-78ca9fa3807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:45.322333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051abd33-4143-4e0b-934b-78ca9fa3807e admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:45.322777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051abd33-4143-4e0b-934b-78ca9fa3807e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:45.324403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051abd33-4143-4e0b-934b-78ca9fa3807e admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:45.325610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051abd33-4143-4e0b-934b-78ca9fa3807e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:45.330053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051abd33-4143-4e0b-934b-78ca9fa3807e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:45.342424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1076/2138] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:45.356722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ea4bf-ea0f-4434-b91c-84990cdafc2c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:45.371212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ea4bf-ea0f-4434-b91c-84990cdafc2c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:45.378241 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665f93e-1278-4aa9-9fd6-ff626db40d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:45.379200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665f93e-1278-4aa9-9fd6-ff626db40d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:45.380214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665f93e-1278-4aa9-9fd6-ff626db40d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:45.380614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665f93e-1278-4aa9-9fd6-ff626db40d43 admin admin] PATH_INFO: `/v3/projects/44cd59484bf54f36816db7a0576a1914/users/c4420fd8171d4768829802bc6c6b3dea/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:45.402333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665f93e-1278-4aa9-9fd6-ff626db40d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cd59484bf54f36816db7a0576a1914, user_id=c4420fd8171d4768829802bc6c6b3dea, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:45.405134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665f93e-1278-4aa9-9fd6-ff626db40d43 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:45.483226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1063/2139] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:45 2022] PUT /identity/v3/projects/44cd59484bf54f36816db7a0576a1914/users/c4420fd8171d4768829802bc6c6b3dea/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:45.493906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e501204-2d95-4607-8d5c-dd24ba9214b6 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:45.494215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e501204-2d95-4607-8d5c-dd24ba9214b6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:45.494491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e501204-2d95-4607-8d5c-dd24ba9214b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:45.534705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6e501204-2d95-4607-8d5c-dd24ba9214b6 None None] MFA Rules not processed for user `c4420fd8171d4768829802bc6c6b3dea`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:45.536077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6e501204-2d95-4607-8d5c-dd24ba9214b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:45.606961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1077/2140] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:45.611743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665f93e-1278-4aa9-9fd6-ff626db40d43 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:45.675635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665f93e-1278-4aa9-9fd6-ff626db40d43 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:45.680429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:45.681094 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcaeea-ef2a-43bd-9955-d958f5ce3033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:45.681700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcaeea-ef2a-43bd-9955-d958f5ce3033 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:45.681975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcaeea-ef2a-43bd-9955-d958f5ce3033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:45.682243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcaeea-ef2a-43bd-9955-d958f5ce3033 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:45.682940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcaeea-ef2a-43bd-9955-d958f5ce3033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:45.684886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcaeea-ef2a-43bd-9955-d958f5ce3033 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:45.805607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1064/2141] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:45 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:45.811458 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e501204-2d95-4607-8d5c-dd24ba9214b6 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:45.820884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:45.828352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e501204-2d95-4607-8d5c-dd24ba9214b6 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:45.953062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3168e-ad1b-433a-ac65-d202ec596374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:45.953855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3168e-ad1b-433a-ac65-d202ec596374 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:45.954217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3168e-ad1b-433a-ac65-d202ec596374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:45.954587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3168e-ad1b-433a-ac65-d202ec596374 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:45.956649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3168e-ad1b-433a-ac65-d202ec596374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:45.961069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3168e-ad1b-433a-ac65-d202ec596374 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:46.096074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1078/2142] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:45 2022] POST /identity/v3/users => generated 296 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:46.097122 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcaeea-ef2a-43bd-9955-d958f5ce3033 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:46.118052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:46.121655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcaeea-ef2a-43bd-9955-d958f5ce3033 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:46.136711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a11f75-e87d-4cde-a714-12eaccd98c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:46.139807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a11f75-e87d-4cde-a714-12eaccd98c27 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:46.141136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a11f75-e87d-4cde-a714-12eaccd98c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:46.141811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a11f75-e87d-4cde-a714-12eaccd98c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:46.142654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a11f75-e87d-4cde-a714-12eaccd98c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:46.147168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a11f75-e87d-4cde-a714-12eaccd98c27 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:46.181207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1065/2143] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:46.187630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3168e-ad1b-433a-ac65-d202ec596374 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:46.204671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3168e-ad1b-433a-ac65-d202ec596374 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:46.210504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c26df8-fb0f-4dc0-9c5f-d8fa40272fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:46.211184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26df8-fb0f-4dc0-9c5f-d8fa40272fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:46.211493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26df8-fb0f-4dc0-9c5f-d8fa40272fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:46.211801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26df8-fb0f-4dc0-9c5f-d8fa40272fe4 admin admin] PATH_INFO: `/v3/projects/f015506b53f1407594866693139a553e/users/013167e8f2d04f74b85b9d54420c1d1f/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:46.229999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c26df8-fb0f-4dc0-9c5f-d8fa40272fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f015506b53f1407594866693139a553e, user_id=013167e8f2d04f74b85b9d54420c1d1f, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:46.232775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c26df8-fb0f-4dc0-9c5f-d8fa40272fe4 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:46.349878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1079/2144] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:46 2022] PUT /identity/v3/projects/f015506b53f1407594866693139a553e/users/013167e8f2d04f74b85b9d54420c1d1f/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:46.354211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a11f75-e87d-4cde-a714-12eaccd98c27 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:46.368419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a11f75-e87d-4cde-a714-12eaccd98c27 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:46.428884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaf7d0-5d75-40db-bf30-78cf326874b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:46.429533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaf7d0-5d75-40db-bf30-78cf326874b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:46.429922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaf7d0-5d75-40db-bf30-78cf326874b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:46.430357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaf7d0-5d75-40db-bf30-78cf326874b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:46.431017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeaf7d0-5d75-40db-bf30-78cf326874b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:46.432756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeaf7d0-5d75-40db-bf30-78cf326874b6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:46.443769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1066/2145] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:46.447936 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c26df8-fb0f-4dc0-9c5f-d8fa40272fe4 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:46.460878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c26df8-fb0f-4dc0-9c5f-d8fa40272fe4 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:46.466022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408a37a-fd2e-4f80-a532-500b7cd078d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:46.466830 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408a37a-fd2e-4f80-a532-500b7cd078d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:46.467203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408a37a-fd2e-4f80-a532-500b7cd078d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:46.467730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408a37a-fd2e-4f80-a532-500b7cd078d3 admin admin] PATH_INFO: `/v3/projects/f015506b53f1407594866693139a553e/users/013167e8f2d04f74b85b9d54420c1d1f/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:46.469821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408a37a-fd2e-4f80-a532-500b7cd078d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f015506b53f1407594866693139a553e, user_id=013167e8f2d04f74b85b9d54420c1d1f, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:46.472770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408a37a-fd2e-4f80-a532-500b7cd078d3 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:46.484597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1080/2146] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:46 2022] PUT /identity/v3/projects/f015506b53f1407594866693139a553e/users/013167e8f2d04f74b85b9d54420c1d1f/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:46.495518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ba4b2-9ac4-41f3-9cba-ed82651e78be None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:46.496040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ba4b2-9ac4-41f3-9cba-ed82651e78be None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:46.496331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ba4b2-9ac4-41f3-9cba-ed82651e78be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:46.526608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b99ba4b2-9ac4-41f3-9cba-ed82651e78be None None] MFA Rules not processed for user `013167e8f2d04f74b85b9d54420c1d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:46.528076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b99ba4b2-9ac4-41f3-9cba-ed82651e78be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:46.550680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:46.605595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1067/2147] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:46.614506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408a37a-fd2e-4f80-a532-500b7cd078d3 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:46.616329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9408a37a-fd2e-4f80-a532-500b7cd078d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:46.677528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408a37a-fd2e-4f80-a532-500b7cd078d3 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:46.681590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013c47e-f874-4a4a-b9e6-478ce8630fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:46.682222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013c47e-f874-4a4a-b9e6-478ce8630fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:46.682506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013c47e-f874-4a4a-b9e6-478ce8630fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:46.682779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013c47e-f874-4a4a-b9e6-478ce8630fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:46.692934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013c47e-f874-4a4a-b9e6-478ce8630fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:46.694383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013c47e-f874-4a4a-b9e6-478ce8630fd5 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:46.703945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1081/2148] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:46.835778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ba4b2-9ac4-41f3-9cba-ed82651e78be None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:46.837568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b99ba4b2-9ac4-41f3-9cba-ed82651e78be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:46.916118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ba4b2-9ac4-41f3-9cba-ed82651e78be None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:46.921312 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:46.922078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:46.922357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:46.922634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] PATH_INFO: `/v3/users/45e72a1cad344482ad5889e7845952ac` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:46.923785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e72a1cad344482ad5889e7845952ac)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:46.925791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:47.064103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:47.195263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e72a1cad344482ad5889e7845952ac'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:47.196029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e72a1cad344482ad5889e7845952ac'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:47.329663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e72a1cad344482ad5889e7845952ac'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:47.347080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e72a1cad344482ad5889e7845952ac'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:47.348091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e72a1cad344482ad5889e7845952ac'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:47.371444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1068/2149] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:46 2022] DELETE /identity/v3/users/45e72a1cad344482ad5889e7845952ac => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:47.376596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb8e45-3c54-432c-8595-89f7addd5525 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:47.376869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb8e45-3c54-432c-8595-89f7addd5525 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:47.377139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb8e45-3c54-432c-8595-89f7addd5525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:47.395359 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:47.398539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:47.402004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-67bb8e45-3c54-432c-8595-89f7addd5525 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:47.403200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-67bb8e45-3c54-432c-8595-89f7addd5525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:47.499387 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1082/2150] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:47.511627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78c08e-3cca-4c89-968b-efd6aa8a3f7d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:47.517740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb8e45-3c54-432c-8595-89f7addd5525 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:47.518809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:47.520489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:47.520489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:47.520489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] PATH_INFO: `/v3/users/c4420fd8171d4768829802bc6c6b3dea` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:47.520849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-67bb8e45-3c54-432c-8595-89f7addd5525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:47.521675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4420fd8171d4768829802bc6c6b3dea)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:47.523979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:47.618595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb8e45-3c54-432c-8595-89f7addd5525 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:47.623650 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3aad6-5033-4930-bc69-86a6dc4c46d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:47.623650 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3aad6-5033-4930-bc69-86a6dc4c46d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:47.623650 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3aad6-5033-4930-bc69-86a6dc4c46d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:47.623997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3aad6-5033-4930-bc69-86a6dc4c46d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:47.635026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3aad6-5033-4930-bc69-86a6dc4c46d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:47.636621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3aad6-5033-4930-bc69-86a6dc4c46d2 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:47.647997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1083/2151] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:47.793575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4420fd8171d4768829802bc6c6b3dea'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:47.794401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4420fd8171d4768829802bc6c6b3dea'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:47.823867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:47.848493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4420fd8171d4768829802bc6c6b3dea'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:47.863454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4420fd8171d4768829802bc6c6b3dea'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:47.864388 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4420fd8171d4768829802bc6c6b3dea'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:47.877665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1069/2152] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:47 2022] DELETE /identity/v3/users/c4420fd8171d4768829802bc6c6b3dea => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:47.883734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940add34-9aa2-4911-bb2b-0d37b684d985 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:47.884020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940add34-9aa2-4911-bb2b-0d37b684d985 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:47.884311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940add34-9aa2-4911-bb2b-0d37b684d985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:47.908926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-940add34-9aa2-4911-bb2b-0d37b684d985 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:47.910212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-940add34-9aa2-4911-bb2b-0d37b684d985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:47.986438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1084/2153] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:47.996758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:47.999389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:48.077868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60323d-e981-406b-bf1f-c96af50d3643 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:48.082259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e61032-480e-442a-a2a6-e34dbada94a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:48.083434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e61032-480e-442a-a2a6-e34dbada94a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:48.083891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e61032-480e-442a-a2a6-e34dbada94a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:48.084314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e61032-480e-442a-a2a6-e34dbada94a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:48.095061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e61032-480e-442a-a2a6-e34dbada94a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:48.096529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e61032-480e-442a-a2a6-e34dbada94a6 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:48.107076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1070/2154] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:48.157509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:48.193923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940add34-9aa2-4911-bb2b-0d37b684d985 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:48.198780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-940add34-9aa2-4911-bb2b-0d37b684d985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:48.210430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940add34-9aa2-4911-bb2b-0d37b684d985 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:48.216385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:48.217037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:48.217318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:48.217592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] PATH_INFO: `/v3/projects/1d1c7062fcf945ce9ef2b1a57d436fca` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:48.224391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d1c7062fcf945ce9ef2b1a57d436fca)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:48.236281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:48.440993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d1c7062fcf945ce9ef2b1a57d436fca', 'request_id': 'req-7d171a32-7449-40a8-928d-769af2ab8795'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:48.451033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d1c7062fcf945ce9ef2b1a57d436fca', 'request_id': 'req-7d171a32-7449-40a8-928d-769af2ab8795'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:48.546701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d1c7062fcf945ce9ef2b1a57d436fca', 'request_id': 'req-7d171a32-7449-40a8-928d-769af2ab8795'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:48.574401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] The token cache is being invalidate because project 1d1c7062fcf945ce9ef2b1a57d436fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:48.574798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:48.575704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:48.579623 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1085/2155] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:48 2022] DELETE /identity/v3/projects/1d1c7062fcf945ce9ef2b1a57d436fca => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:48.591611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a9ff-3a10-4458-803a-c303fe329045 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:48.592434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a9ff-3a10-4458-803a-c303fe329045 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:48.592663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a9ff-3a10-4458-803a-c303fe329045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:48.618595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3542a9ff-3a10-4458-803a-c303fe329045 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:48.620099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3542a9ff-3a10-4458-803a-c303fe329045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:48.687772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:48.689651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:48.695936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1071/2156] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:48.703154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d171a32-7449-40a8-928d-769af2ab8795 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:48.709188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:48.710133 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:48.714900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:48.714900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] PATH_INFO: `/v3/projects/44cd59484bf54f36816db7a0576a1914` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:48.715686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c764d4c-95e8-4628-ae07-90b0d294b5a9 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:48.715686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c764d4c-95e8-4628-ae07-90b0d294b5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:48.715686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c764d4c-95e8-4628-ae07-90b0d294b5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:48.716038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44cd59484bf54f36816db7a0576a1914)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:48.716038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:48.735781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5c764d4c-95e8-4628-ae07-90b0d294b5a9 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:48.736983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5c764d4c-95e8-4628-ae07-90b0d294b5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:48.746875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1072/2157] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:48.751136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c764d4c-95e8-4628-ae07-90b0d294b5a9 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:48.760799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c764d4c-95e8-4628-ae07-90b0d294b5a9 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:48.763815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:48.768957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3e224-7d08-43e6-aaf4-96da7f9f5f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:48.769698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3e224-7d08-43e6-aaf4-96da7f9f5f3d admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:48.770074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3e224-7d08-43e6-aaf4-96da7f9f5f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:48.770924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3e224-7d08-43e6-aaf4-96da7f9f5f3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:48.771715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3e224-7d08-43e6-aaf4-96da7f9f5f3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:48.772099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3e224-7d08-43e6-aaf4-96da7f9f5f3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:48.774144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3e224-7d08-43e6-aaf4-96da7f9f5f3d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:48.787597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1073/2158] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:30:48.792103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3e224-7d08-43e6-aaf4-96da7f9f5f3d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:48.802366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3e224-7d08-43e6-aaf4-96da7f9f5f3d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:48.807085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ba911-20f7-4f1d-8e11-3874cf49ff70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:48.807724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ba911-20f7-4f1d-8e11-3874cf49ff70 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:48.808002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ba911-20f7-4f1d-8e11-3874cf49ff70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:48.808270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ba911-20f7-4f1d-8e11-3874cf49ff70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:48.808896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ba911-20f7-4f1d-8e11-3874cf49ff70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:48.810785 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ba911-20f7-4f1d-8e11-3874cf49ff70 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:48.840833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44cd59484bf54f36816db7a0576a1914', 'request_id': 'req-006f5631-2763-405c-a397-299ba61cca27'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:48.846269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44cd59484bf54f36816db7a0576a1914', 'request_id': 'req-006f5631-2763-405c-a397-299ba61cca27'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:48.912304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1074/2159] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:48 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:48.915188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ba911-20f7-4f1d-8e11-3874cf49ff70 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:48.928863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44cd59484bf54f36816db7a0576a1914', 'request_id': 'req-006f5631-2763-405c-a397-299ba61cca27'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:48.947637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] The token cache is being invalidate because project 44cd59484bf54f36816db7a0576a1914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:48.947637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:48.947637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:48.947637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1086/2160] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:48 2022] DELETE /identity/v3/projects/44cd59484bf54f36816db7a0576a1914 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:48.950623 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:48.953320 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:48.997453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ba911-20f7-4f1d-8e11-3874cf49ff70 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:49.002689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ef763-b410-40f2-a400-38fed56956ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:49.003501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ef763-b410-40f2-a400-38fed56956ec admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:49.004055 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ef763-b410-40f2-a400-38fed56956ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:49.004359 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ef763-b410-40f2-a400-38fed56956ec admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:49.005244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ef763-b410-40f2-a400-38fed56956ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:49.007631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ef763-b410-40f2-a400-38fed56956ec admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:49.036720 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f5631-2763-405c-a397-299ba61cca27 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:49.041467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:49.042120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:49.042395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:49.042671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] PATH_INFO: `/v3/users/013167e8f2d04f74b85b9d54420c1d1f` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:49.043875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=013167e8f2d04f74b85b9d54420c1d1f)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:49.045838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:49.091815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1075/2161] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:48 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:49.095998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ef763-b410-40f2-a400-38fed56956ec admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:49.097668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-781ef763-b410-40f2-a400-38fed56956ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:49.110689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ef763-b410-40f2-a400-38fed56956ec admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:49.116385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd78c6-3a1f-4250-a606-4594c4c7efc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:49.116997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd78c6-3a1f-4250-a606-4594c4c7efc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:49.117275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd78c6-3a1f-4250-a606-4594c4c7efc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:49.117547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd78c6-3a1f-4250-a606-4594c4c7efc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:49.118184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd78c6-3a1f-4250-a606-4594c4c7efc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:49.119874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd78c6-3a1f-4250-a606-4594c4c7efc0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:49.130216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1076/2162] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:49.134136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd78c6-3a1f-4250-a606-4594c4c7efc0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:49.144393 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd78c6-3a1f-4250-a606-4594c4c7efc0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:49.149147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bb98c-1b61-4e7e-937c-38a54cf11217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:49.149746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bb98c-1b61-4e7e-937c-38a54cf11217 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:49.150032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bb98c-1b61-4e7e-937c-38a54cf11217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:49.150354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bb98c-1b61-4e7e-937c-38a54cf11217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:49.150949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bb98c-1b61-4e7e-937c-38a54cf11217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:49.152624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bb98c-1b61-4e7e-937c-38a54cf11217 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:49.164018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1077/2163] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:49.168135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bb98c-1b61-4e7e-937c-38a54cf11217 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:49.178610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bb98c-1b61-4e7e-937c-38a54cf11217 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:49.183741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731e656-1bc9-438b-ac61-728af81f3e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:49.184408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e656-1bc9-438b-ac61-728af81f3e08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:49.184748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e656-1bc9-438b-ac61-728af81f3e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:49.185026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e656-1bc9-438b-ac61-728af81f3e08 admin admin] PATH_INFO: `/v3/projects/2f977c80e7514de8ae6bef0df5ee953b/users/6be49bc260dc41f4993546f55174c18c/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:49.200471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731e656-1bc9-438b-ac61-728af81f3e08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f977c80e7514de8ae6bef0df5ee953b, user_id=6be49bc260dc41f4993546f55174c18c, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:49.204581 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731e656-1bc9-438b-ac61-728af81f3e08 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:49.257243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '013167e8f2d04f74b85b9d54420c1d1f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:49.257930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '013167e8f2d04f74b85b9d54420c1d1f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:49.278596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1078/2164] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:49 2022] PUT /identity/v3/projects/2f977c80e7514de8ae6bef0df5ee953b/users/6be49bc260dc41f4993546f55174c18c/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:49.289751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8809aee-fa69-45c7-bd5c-7fdafb990b99 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:49.290040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8809aee-fa69-45c7-bd5c-7fdafb990b99 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:49.290319 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8809aee-fa69-45c7-bd5c-7fdafb990b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:49.319194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b8809aee-fa69-45c7-bd5c-7fdafb990b99 None None] MFA Rules not processed for user `6be49bc260dc41f4993546f55174c18c`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:49.320364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b8809aee-fa69-45c7-bd5c-7fdafb990b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:49.334170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '013167e8f2d04f74b85b9d54420c1d1f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:49.349244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '013167e8f2d04f74b85b9d54420c1d1f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:49.350072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '013167e8f2d04f74b85b9d54420c1d1f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:49.362954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1087/2165] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:48 2022] DELETE /identity/v3/users/013167e8f2d04f74b85b9d54420c1d1f => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:49.388653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1079/2166] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:49.400620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:49.402407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:49.425523 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:49.485932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248744d1-b8dc-4bdf-a99e-72aeaef48d1d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:49.491932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebe030-7acf-444c-8e7e-5aec280a2e1f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '237bbeebe2a949aba444353c1d2ee04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:49.492923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebe030-7acf-444c-8e7e-5aec280a2e1f service glance] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:49.493395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebe030-7acf-444c-8e7e-5aec280a2e1f service glance] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:49.493852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebe030-7acf-444c-8e7e-5aec280a2e1f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:49.525945 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:49.556928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebe030-7acf-444c-8e7e-5aec280a2e1f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:49.562766 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebe030-7acf-444c-8e7e-5aec280a2e1f service glance] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:49.577034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1088/2167] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:30:49.854650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8809aee-fa69-45c7-bd5c-7fdafb990b99 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:49.856099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b8809aee-fa69-45c7-bd5c-7fdafb990b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:49.978363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8809aee-fa69-45c7-bd5c-7fdafb990b99 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:50.036932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:50.037628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:50.037908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:50.038187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] PATH_INFO: `/v3/projects/f015506b53f1407594866693139a553e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:50.039355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f015506b53f1407594866693139a553e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:50.049738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:50.094700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:50.179453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f015506b53f1407594866693139a553e', 'request_id': 'req-51b42632-8e4c-4463-9c62-3bf25d5b6972'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:50.185763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f015506b53f1407594866693139a553e', 'request_id': 'req-51b42632-8e4c-4463-9c62-3bf25d5b6972'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:50.277915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:50.278462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f015506b53f1407594866693139a553e', 'request_id': 'req-51b42632-8e4c-4463-9c62-3bf25d5b6972'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:50.291117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] The token cache is being invalidate because project f015506b53f1407594866693139a553e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:50.291408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:50.292114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-51b42632-8e4c-4463-9c62-3bf25d5b6972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:50.293932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1080/2168] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:49 2022] DELETE /identity/v3/projects/f015506b53f1407594866693139a553e => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:50.300938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d524e-f15b-4bce-b521-af1f20cf3323 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:50.301202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d524e-f15b-4bce-b521-af1f20cf3323 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:50.301566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d524e-f15b-4bce-b521-af1f20cf3323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:50.322639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-4f5d524e-f15b-4bce-b521-af1f20cf3323 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:50.323908 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4f5d524e-f15b-4bce-b521-af1f20cf3323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:50.387687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1089/2169] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:50.396557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc720a-1417-4b8d-8b16-73a82992c7e0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:50.396843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc720a-1417-4b8d-8b16-73a82992c7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:50.397157 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc720a-1417-4b8d-8b16-73a82992c7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:50.418115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-21cc720a-1417-4b8d-8b16-73a82992c7e0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:50.419339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-21cc720a-1417-4b8d-8b16-73a82992c7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:50.427580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1081/2170] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:50.431984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d524e-f15b-4bce-b521-af1f20cf3323 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:50.441893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d524e-f15b-4bce-b521-af1f20cf3323 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:50.446674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c574a1-62d2-4531-aa6b-90322e834d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:50.447304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c574a1-62d2-4531-aa6b-90322e834d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:50.447727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c574a1-62d2-4531-aa6b-90322e834d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:50.448008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c574a1-62d2-4531-aa6b-90322e834d6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:50.448668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c574a1-62d2-4531-aa6b-90322e834d6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:50.459041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c574a1-62d2-4531-aa6b-90322e834d6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:50.459041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c574a1-62d2-4531-aa6b-90322e834d6a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:50.463439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1090/2171] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:50.468013 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc720a-1417-4b8d-8b16-73a82992c7e0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:50.477950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc720a-1417-4b8d-8b16-73a82992c7e0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:50.483291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9d683-b61d-44ee-b675-78d44e46db11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:50.483941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9d683-b61d-44ee-b675-78d44e46db11 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:50.484219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9d683-b61d-44ee-b675-78d44e46db11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:50.484490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9d683-b61d-44ee-b675-78d44e46db11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:50.485127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9d683-b61d-44ee-b675-78d44e46db11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:50.487042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9d683-b61d-44ee-b675-78d44e46db11 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:50.578858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1082/2172] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:50 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:50.582879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c574a1-62d2-4531-aa6b-90322e834d6a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:50.594592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c574a1-62d2-4531-aa6b-90322e834d6a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:50.658777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77841f09-1195-4304-8c70-6486619864eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:50.659516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77841f09-1195-4304-8c70-6486619864eb admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:50.659896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77841f09-1195-4304-8c70-6486619864eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:50.660255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77841f09-1195-4304-8c70-6486619864eb admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:50.661515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77841f09-1195-4304-8c70-6486619864eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:50.663573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77841f09-1195-4304-8c70-6486619864eb admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:50.731496 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1091/2173] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:50 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:50.731847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9d683-b61d-44ee-b675-78d44e46db11 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:50.743194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9d683-b61d-44ee-b675-78d44e46db11 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:50.749042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550131e-5166-4d2d-97f5-e4ece0fc7820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:50.749732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550131e-5166-4d2d-97f5-e4ece0fc7820 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:50.750097 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550131e-5166-4d2d-97f5-e4ece0fc7820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:50.750470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550131e-5166-4d2d-97f5-e4ece0fc7820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:50.751312 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550131e-5166-4d2d-97f5-e4ece0fc7820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:50.753285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550131e-5166-4d2d-97f5-e4ece0fc7820 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:50.764952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1083/2174] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:50.771845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77841f09-1195-4304-8c70-6486619864eb admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:50.779411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77841f09-1195-4304-8c70-6486619864eb admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:50.783596 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:50.784614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78710129-eb93-4f32-b1f6-3bb7f77bff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:50.785352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78710129-eb93-4f32-b1f6-3bb7f77bff86 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:50.785593 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78710129-eb93-4f32-b1f6-3bb7f77bff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:50.786342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78710129-eb93-4f32-b1f6-3bb7f77bff86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:50.786992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78710129-eb93-4f32-b1f6-3bb7f77bff86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:50.788648 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78710129-eb93-4f32-b1f6-3bb7f77bff86 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:50.799868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1092/2175] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:50.806112 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550131e-5166-4d2d-97f5-e4ece0fc7820 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:50.816390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550131e-5166-4d2d-97f5-e4ece0fc7820 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:50.821241 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fffd2c-0e7e-4dcb-8d9a-ed4f41f988ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:50.821899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fffd2c-0e7e-4dcb-8d9a-ed4f41f988ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:50.822176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fffd2c-0e7e-4dcb-8d9a-ed4f41f988ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:50.822449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fffd2c-0e7e-4dcb-8d9a-ed4f41f988ef admin admin] PATH_INFO: `/v3/projects/4377cb4a93524abd99acb6ebe07fd1d6/users/e7a29212dfae4a93beea7d11085cde63/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:50.837667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:50.837873 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fffd2c-0e7e-4dcb-8d9a-ed4f41f988ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=4377cb4a93524abd99acb6ebe07fd1d6, user_id=e7a29212dfae4a93beea7d11085cde63, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:50.840552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fffd2c-0e7e-4dcb-8d9a-ed4f41f988ef admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:50.953840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1084/2176] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:50 2022] PUT /identity/v3/projects/4377cb4a93524abd99acb6ebe07fd1d6/users/e7a29212dfae4a93beea7d11085cde63/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:50.959706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373877d1-924b-460d-b1c9-7bbd3d8f30d4 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:50.960023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373877d1-924b-460d-b1c9-7bbd3d8f30d4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:50.960271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373877d1-924b-460d-b1c9-7bbd3d8f30d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:50.983994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-373877d1-924b-460d-b1c9-7bbd3d8f30d4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:50.985241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-373877d1-924b-460d-b1c9-7bbd3d8f30d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:51.054882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1093/2177] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:51.065529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fffd2c-0e7e-4dcb-8d9a-ed4f41f988ef admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:51.071019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-37fffd2c-0e7e-4dcb-8d9a-ed4f41f988ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:51.133893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fffd2c-0e7e-4dcb-8d9a-ed4f41f988ef admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:51.146506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03493d-d6f1-45d5-a763-7a40092ccb18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:51.146506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03493d-d6f1-45d5-a763-7a40092ccb18 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:51.146506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03493d-d6f1-45d5-a763-7a40092ccb18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:51.146506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03493d-d6f1-45d5-a763-7a40092ccb18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:51.149465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03493d-d6f1-45d5-a763-7a40092ccb18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:51.150913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03493d-d6f1-45d5-a763-7a40092ccb18 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:51.162568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1085/2178] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:30:51.197503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:51.274450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:51.605114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:51.684555 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:51.808658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373877d1-924b-460d-b1c9-7bbd3d8f30d4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:51.810211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-373877d1-924b-460d-b1c9-7bbd3d8f30d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:51.824215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373877d1-924b-460d-b1c9-7bbd3d8f30d4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:51.829317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:51.829956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:51.830598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:51.830598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] PATH_INFO: `/v3/users/ba446852f7c644ed9f55d99aa7edcd19` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:51.832094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba446852f7c644ed9f55d99aa7edcd19)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:51.834097 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.052240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:52.100307 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba446852f7c644ed9f55d99aa7edcd19'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:52.101019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba446852f7c644ed9f55d99aa7edcd19'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:52.200367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba446852f7c644ed9f55d99aa7edcd19'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:52.221434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba446852f7c644ed9f55d99aa7edcd19'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:52.221633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba446852f7c644ed9f55d99aa7edcd19'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:52.243501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1094/2179] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:51 2022] DELETE /identity/v3/users/ba446852f7c644ed9f55d99aa7edcd19 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:52.247837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243ec35-56a8-4921-8381-0317fdbe9615 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.248335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243ec35-56a8-4921-8381-0317fdbe9615 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.250183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243ec35-56a8-4921-8381-0317fdbe9615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.273087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e243ec35-56a8-4921-8381-0317fdbe9615 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:52.274685 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e243ec35-56a8-4921-8381-0317fdbe9615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:52.286291 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=6 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:52.352935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1086/2180] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:52.363136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:52.364935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:52.440351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95d8dd-e0d9-4f70-8723-2d767668c2a2 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:52.445013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa537e-0f82-4cf4-8279-687cd90a337b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:52.445644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa537e-0f82-4cf4-8279-687cd90a337b service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:52.445926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa537e-0f82-4cf4-8279-687cd90a337b service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:52.446223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa537e-0f82-4cf4-8279-687cd90a337b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:52.459515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa537e-0f82-4cf4-8279-687cd90a337b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:52.461279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa537e-0f82-4cf4-8279-687cd90a337b service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:52.472383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1095/2181] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:53.057898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243ec35-56a8-4921-8381-0317fdbe9615 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:53.062569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e243ec35-56a8-4921-8381-0317fdbe9615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:53.088497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243ec35-56a8-4921-8381-0317fdbe9615 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:53.097667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:53.099391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:53.099788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:53.100342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] PATH_INFO: `/v3/projects/53455294ed464787a602cd30df706f7c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:53.102123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53455294ed464787a602cd30df706f7c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:53.105076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:53.263773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53455294ed464787a602cd30df706f7c', 'request_id': 'req-77a49653-ab3f-47c4-a972-abf9e9e00da8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:53.269340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53455294ed464787a602cd30df706f7c', 'request_id': 'req-77a49653-ab3f-47c4-a972-abf9e9e00da8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:53.325427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:53.363215 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53455294ed464787a602cd30df706f7c', 'request_id': 'req-77a49653-ab3f-47c4-a972-abf9e9e00da8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:53.376862 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] The token cache is being invalidate because project 53455294ed464787a602cd30df706f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:53.381553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:53.382540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77a49653-ab3f-47c4-a972-abf9e9e00da8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:53.384445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1087/2182] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:53 2022] DELETE /identity/v3/projects/53455294ed464787a602cd30df706f7c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:53.389043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf39d51-afec-40cd-a3fd-76927f46c4fa None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:53.389555 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf39d51-afec-40cd-a3fd-76927f46c4fa None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:53.389666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf39d51-afec-40cd-a3fd-76927f46c4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:53.421802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-5bf39d51-afec-40cd-a3fd-76927f46c4fa None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:53.423035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5bf39d51-afec-40cd-a3fd-76927f46c4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:53.505274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1096/2183] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:53.513042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8832a8-78be-47e0-91d4-ff0e0aa9ec16 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:53.513927 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8832a8-78be-47e0-91d4-ff0e0aa9ec16 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:53.514390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8832a8-78be-47e0-91d4-ff0e0aa9ec16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:53.543841 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3b8832a8-78be-47e0-91d4-ff0e0aa9ec16 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:53.545219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3b8832a8-78be-47e0-91d4-ff0e0aa9ec16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:53.555338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1088/2184] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:53.560384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf39d51-afec-40cd-a3fd-76927f46c4fa None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:53.571694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf39d51-afec-40cd-a3fd-76927f46c4fa None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:53.577988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:53.579690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834ef7c-749f-4762-9fd5-f22a76b1d76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:53.580591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834ef7c-749f-4762-9fd5-f22a76b1d76b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:53.580990 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834ef7c-749f-4762-9fd5-f22a76b1d76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:53.581400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834ef7c-749f-4762-9fd5-f22a76b1d76b admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:53.582070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834ef7c-749f-4762-9fd5-f22a76b1d76b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:53.582487 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834ef7c-749f-4762-9fd5-f22a76b1d76b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:53.584812 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834ef7c-749f-4762-9fd5-f22a76b1d76b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:53.598492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1097/2185] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:53.602722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8832a8-78be-47e0-91d4-ff0e0aa9ec16 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:53.612922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8832a8-78be-47e0-91d4-ff0e0aa9ec16 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:53.621135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc6b78-8afc-45af-8198-898a7e82effb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:53.621762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc6b78-8afc-45af-8198-898a7e82effb admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:53.622045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc6b78-8afc-45af-8198-898a7e82effb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:53.622315 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc6b78-8afc-45af-8198-898a7e82effb admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:53.622999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc6b78-8afc-45af-8198-898a7e82effb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:53.624953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc6b78-8afc-45af-8198-898a7e82effb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:53.721527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1089/2186] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:53 2022] POST /identity/v3/projects => generated 359 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:53.725890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834ef7c-749f-4762-9fd5-f22a76b1d76b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:53.737271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834ef7c-749f-4762-9fd5-f22a76b1d76b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:53.819668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33e8b9-5c70-41d3-b7a7-a8dda3adfc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:53.819668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33e8b9-5c70-41d3-b7a7-a8dda3adfc29 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:53.819668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33e8b9-5c70-41d3-b7a7-a8dda3adfc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:53.819668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33e8b9-5c70-41d3-b7a7-a8dda3adfc29 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:53.819975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33e8b9-5c70-41d3-b7a7-a8dda3adfc29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:53.823301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33e8b9-5c70-41d3-b7a7-a8dda3adfc29 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:53.963815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1098/2187] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:53 2022] POST /identity/v3/users => generated 280 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:53.967725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc6b78-8afc-45af-8198-898a7e82effb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:53.993745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc6b78-8afc-45af-8198-898a7e82effb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:53.998678 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43093e6-382a-4a1c-bbaa-539ba0465dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:53.999475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43093e6-382a-4a1c-bbaa-539ba0465dad admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:53.999982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43093e6-382a-4a1c-bbaa-539ba0465dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:54.000417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43093e6-382a-4a1c-bbaa-539ba0465dad admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:54.001115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43093e6-382a-4a1c-bbaa-539ba0465dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:54.002750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43093e6-382a-4a1c-bbaa-539ba0465dad admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:54.015598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1090/2188] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:54.020910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33e8b9-5c70-41d3-b7a7-a8dda3adfc29 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:54.031483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33e8b9-5c70-41d3-b7a7-a8dda3adfc29 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:54.038748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935da64e-5cf8-4eac-b2ff-83e69eecfde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:54.039374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935da64e-5cf8-4eac-b2ff-83e69eecfde0 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:54.039997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935da64e-5cf8-4eac-b2ff-83e69eecfde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:54.040285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935da64e-5cf8-4eac-b2ff-83e69eecfde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:54.040937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935da64e-5cf8-4eac-b2ff-83e69eecfde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:54.042579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935da64e-5cf8-4eac-b2ff-83e69eecfde0 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:54.054807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1099/2189] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:54.059096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43093e6-382a-4a1c-bbaa-539ba0465dad admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:54.069982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43093e6-382a-4a1c-bbaa-539ba0465dad admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:54.075417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbaf72-cd0e-482c-bf88-3410b2fb1410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:54.076138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbaf72-cd0e-482c-bf88-3410b2fb1410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:54.076421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbaf72-cd0e-482c-bf88-3410b2fb1410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:54.077046 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbaf72-cd0e-482c-bf88-3410b2fb1410 admin admin] PATH_INFO: `/v3/projects/cbac6664fdac4ce2aa8e8034103f974c/users/1cfe6b89060246ddb33ab192733340f8/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:54.104556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbaf72-cd0e-482c-bf88-3410b2fb1410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbac6664fdac4ce2aa8e8034103f974c, user_id=1cfe6b89060246ddb33ab192733340f8, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:54.107761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbaf72-cd0e-482c-bf88-3410b2fb1410 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:54.264936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1091/2190] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:54 2022] PUT /identity/v3/projects/cbac6664fdac4ce2aa8e8034103f974c/users/1cfe6b89060246ddb33ab192733340f8/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:54.275191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfc030-a9ad-47fd-9ee9-64eda6d228ce None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:54.275481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfc030-a9ad-47fd-9ee9-64eda6d228ce None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:54.275801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfc030-a9ad-47fd-9ee9-64eda6d228ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:54.310979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1ecfc030-a9ad-47fd-9ee9-64eda6d228ce None None] MFA Rules not processed for user `1cfe6b89060246ddb33ab192733340f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:54.312266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1ecfc030-a9ad-47fd-9ee9-64eda6d228ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:54.407965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1100/2191] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:54.455874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbaf72-cd0e-482c-bf88-3410b2fb1410 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:54.457708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1ddbaf72-cd0e-482c-bf88-3410b2fb1410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:54.519204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbaf72-cd0e-482c-bf88-3410b2fb1410 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:54.522987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb826f7-6199-4b3c-9ed9-c901b6feb34d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:54.524255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb826f7-6199-4b3c-9ed9-c901b6feb34d service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:54.524255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb826f7-6199-4b3c-9ed9-c901b6feb34d service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:54.524255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb826f7-6199-4b3c-9ed9-c901b6feb34d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:54.532694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb826f7-6199-4b3c-9ed9-c901b6feb34d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:54.534189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb826f7-6199-4b3c-9ed9-c901b6feb34d service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:54.544590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1092/2192] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:54.562622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecfc030-a9ad-47fd-9ee9-64eda6d228ce None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:54.632154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecfc030-a9ad-47fd-9ee9-64eda6d228ce None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:54.636807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:54.637451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:54.637734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:54.638010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] PATH_INFO: `/v3/users/1cfe6b89060246ddb33ab192733340f8` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:54.639211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cfe6b89060246ddb33ab192733340f8)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:54.641426 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:54.689326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:54.859463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cfe6b89060246ddb33ab192733340f8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:54.861198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cfe6b89060246ddb33ab192733340f8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:54.946730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cfe6b89060246ddb33ab192733340f8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:54.977801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cfe6b89060246ddb33ab192733340f8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:54.978638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cfe6b89060246ddb33ab192733340f8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:54.995211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1101/2193] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:54 2022] DELETE /identity/v3/users/1cfe6b89060246ddb33ab192733340f8 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:55.000571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c669747-77a3-4211-957a-eb003af0acfc None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:55.001245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c669747-77a3-4211-957a-eb003af0acfc None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:55.001795 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c669747-77a3-4211-957a-eb003af0acfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:55.024171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:55.026424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4c669747-77a3-4211-957a-eb003af0acfc None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:55.027852 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4c669747-77a3-4211-957a-eb003af0acfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:55.106290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1093/2194] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:55.123890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:55.126328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:55.270379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f960ab8-0b8a-436a-bff2-a8ccbdc24d5b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:55.278918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad63e5b-1858-4d11-955e-537468ce2c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:55.280051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad63e5b-1858-4d11-955e-537468ce2c93 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:55.280468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad63e5b-1858-4d11-955e-537468ce2c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:55.280863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad63e5b-1858-4d11-955e-537468ce2c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:55.298747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad63e5b-1858-4d11-955e-537468ce2c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:55.301637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad63e5b-1858-4d11-955e-537468ce2c93 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:55.323885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1102/2195] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:56.218052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:56.425764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:56.496059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c669747-77a3-4211-957a-eb003af0acfc None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:56.497151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4c669747-77a3-4211-957a-eb003af0acfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:56.523588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c669747-77a3-4211-957a-eb003af0acfc None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:56.541919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:56.542622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:56.543053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:56.543925 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] PATH_INFO: `/v3/projects/cbac6664fdac4ce2aa8e8034103f974c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:56.546661 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbac6664fdac4ce2aa8e8034103f974c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:56.552648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:56.741104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbac6664fdac4ce2aa8e8034103f974c', 'request_id': 'req-ec686be8-2931-43e6-b339-a2b7819b091c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:56.764815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbac6664fdac4ce2aa8e8034103f974c', 'request_id': 'req-ec686be8-2931-43e6-b339-a2b7819b091c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:56.871848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbac6664fdac4ce2aa8e8034103f974c', 'request_id': 'req-ec686be8-2931-43e6-b339-a2b7819b091c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:56.885347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] The token cache is being invalidate because project cbac6664fdac4ce2aa8e8034103f974c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:56.885994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:56.886905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:56.888954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1094/2196] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:56 2022] DELETE /identity/v3/projects/cbac6664fdac4ce2aa8e8034103f974c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:56.895036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61ae0b-4b4a-4ed7-8af3-83a0e5fbf7bb None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:56.895695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61ae0b-4b4a-4ed7-8af3-83a0e5fbf7bb None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:56.896215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61ae0b-4b4a-4ed7-8af3-83a0e5fbf7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:56.920472 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:56.922484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:56.925162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-4b61ae0b-4b4a-4ed7-8af3-83a0e5fbf7bb None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:56.926627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4b61ae0b-4b4a-4ed7-8af3-83a0e5fbf7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:57.068998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1103/2197] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:57.070652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec686be8-2931-43e6-b339-a2b7819b091c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:57.070842 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbf265-7755-48d4-9c02-e0a27722b04b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:57.070842 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbf265-7755-48d4-9c02-e0a27722b04b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:57.071000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:57.071108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbf265-7755-48d4-9c02-e0a27722b04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:57.071196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:57.071196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:57.071196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] PATH_INFO: `/v3/users/592d1e35510743fbb014e68f0e2189d5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:57.071196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=592d1e35510743fbb014e68f0e2189d5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:57.071196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:57.113310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-67fbf265-7755-48d4-9c02-e0a27722b04b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:57.116206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-67fbf265-7755-48d4-9c02-e0a27722b04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:57.126507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1104/2198] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:57.132297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbf265-7755-48d4-9c02-e0a27722b04b None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:57.147995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbf265-7755-48d4-9c02-e0a27722b04b None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:57.153896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86c64f-51f1-4410-8fdd-8363ad892b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:57.154513 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c64f-51f1-4410-8fdd-8363ad892b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:57.154799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c64f-51f1-4410-8fdd-8363ad892b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:57.155078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c64f-51f1-4410-8fdd-8363ad892b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:57.155821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86c64f-51f1-4410-8fdd-8363ad892b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:30:57.156213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86c64f-51f1-4410-8fdd-8363ad892b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:57.157880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86c64f-51f1-4410-8fdd-8363ad892b3a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:57.172353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1105/2199] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:30:57.177975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86c64f-51f1-4410-8fdd-8363ad892b3a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:57.192500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86c64f-51f1-4410-8fdd-8363ad892b3a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:57.200038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff7ee8-c04d-47e6-893c-91c8fb010df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:57.200798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff7ee8-c04d-47e6-893c-91c8fb010df6 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:57.201072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff7ee8-c04d-47e6-893c-91c8fb010df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:57.201329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff7ee8-c04d-47e6-893c-91c8fb010df6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:57.203661 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff7ee8-c04d-47e6-893c-91c8fb010df6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:57.206518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff7ee8-c04d-47e6-893c-91c8fb010df6 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:57.406086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1106/2200] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:57 2022] POST /identity/v3/projects => generated 375 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:57.407867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff7ee8-c04d-47e6-893c-91c8fb010df6 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:57.446209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592d1e35510743fbb014e68f0e2189d5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:57.446209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592d1e35510743fbb014e68f0e2189d5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:57.517571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592d1e35510743fbb014e68f0e2189d5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:57.555196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592d1e35510743fbb014e68f0e2189d5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:57.555196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-bac0b708-11eb-4ec7-b809-e1df83ad5662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592d1e35510743fbb014e68f0e2189d5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:57.557400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff7ee8-c04d-47e6-893c-91c8fb010df6 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:57.575400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e18e3-1b4d-4ed7-adf0-aa90713f9eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:57.576088 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e18e3-1b4d-4ed7-adf0-aa90713f9eeb admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:57.576440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e18e3-1b4d-4ed7-adf0-aa90713f9eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:57.576774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e18e3-1b4d-4ed7-adf0-aa90713f9eeb admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:57.580010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e18e3-1b4d-4ed7-adf0-aa90713f9eeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:57.591216 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e18e3-1b4d-4ed7-adf0-aa90713f9eeb admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:57.619170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1095/2201] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:56 2022] DELETE /identity/v3/users/592d1e35510743fbb014e68f0e2189d5 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:57.622138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90584e-77a3-4827-9002-d5ce525a42c7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:57.623007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90584e-77a3-4827-9002-d5ce525a42c7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:57.623495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90584e-77a3-4827-9002-d5ce525a42c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:57.649409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9a90584e-77a3-4827-9002-d5ce525a42c7 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:57.651958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9a90584e-77a3-4827-9002-d5ce525a42c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:57.715042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1107/2202] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:57 2022] POST /identity/v3/users => generated 288 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:30:57.716912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e18e3-1b4d-4ed7-adf0-aa90713f9eeb admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:57.719761 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-096e18e3-1b4d-4ed7-adf0-aa90713f9eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:57.754549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e18e3-1b4d-4ed7-adf0-aa90713f9eeb admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:57.819992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1096/2203] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:57.830355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90584e-77a3-4827-9002-d5ce525a42c7 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:57.831821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9a90584e-77a3-4827-9002-d5ce525a42c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:57.835117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:57.889257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8f84d-81c2-46b7-a1bd-145edd2edb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:57.893233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8f84d-81c2-46b7-a1bd-145edd2edb22 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:57.893548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:57.894307 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8f84d-81c2-46b7-a1bd-145edd2edb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:57.894677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8f84d-81c2-46b7-a1bd-145edd2edb22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:57.895640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8f84d-81c2-46b7-a1bd-145edd2edb22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:57.900762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8f84d-81c2-46b7-a1bd-145edd2edb22 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:57.924156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1108/2204] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:57.933905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8f84d-81c2-46b7-a1bd-145edd2edb22 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:57.945257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8f84d-81c2-46b7-a1bd-145edd2edb22 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:57.954235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff83ab2-2e8c-4b79-8b81-d2a4dc425374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:57.955225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83ab2-2e8c-4b79-8b81-d2a4dc425374 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:57.955514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83ab2-2e8c-4b79-8b81-d2a4dc425374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:57.956475 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83ab2-2e8c-4b79-8b81-d2a4dc425374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:57.957304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff83ab2-2e8c-4b79-8b81-d2a4dc425374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:57.959729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff83ab2-2e8c-4b79-8b81-d2a4dc425374 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:57.981043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1109/2205] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:30:57.982162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90584e-77a3-4827-9002-d5ce525a42c7 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:57.990754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff83ab2-2e8c-4b79-8b81-d2a4dc425374 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:57.992658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740a2a6-d9ab-491f-9eaa-74e17b4d0f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:57.997015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740a2a6-d9ab-491f-9eaa-74e17b4d0f84 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:57.997015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740a2a6-d9ab-491f-9eaa-74e17b4d0f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:57.997266 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740a2a6-d9ab-491f-9eaa-74e17b4d0f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:58.006880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff83ab2-2e8c-4b79-8b81-d2a4dc425374 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:58.014611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7510a-38f1-4a9b-8017-dde85aad139a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:58.015499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7510a-38f1-4a9b-8017-dde85aad139a admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:58.015954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7510a-38f1-4a9b-8017-dde85aad139a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:58.016330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7510a-38f1-4a9b-8017-dde85aad139a admin admin] PATH_INFO: `/v3/projects/a37d5b89b81045679105900f317baabe/users/381213869d604763aa4b36d5b34960d8/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:58.022979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740a2a6-d9ab-491f-9eaa-74e17b4d0f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:58.024752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740a2a6-d9ab-491f-9eaa-74e17b4d0f84 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:58.037912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7510a-38f1-4a9b-8017-dde85aad139a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37d5b89b81045679105900f317baabe, user_id=381213869d604763aa4b36d5b34960d8, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:58.038838 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1097/2206] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:30:58.041713 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7510a-38f1-4a9b-8017-dde85aad139a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:58.169292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1110/2207] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:57 2022] PUT /identity/v3/projects/a37d5b89b81045679105900f317baabe/users/381213869d604763aa4b36d5b34960d8/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:58.180167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcbaff-fe1e-4ad6-a835-001bfe2bf806 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:58.181043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcbaff-fe1e-4ad6-a835-001bfe2bf806 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:58.181649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcbaff-fe1e-4ad6-a835-001bfe2bf806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:58.225323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b5bcbaff-fe1e-4ad6-a835-001bfe2bf806 None None] MFA Rules not processed for user `381213869d604763aa4b36d5b34960d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:58.227080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b5bcbaff-fe1e-4ad6-a835-001bfe2bf806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:58.339880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1098/2208] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:30:58.351350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7510a-38f1-4a9b-8017-dde85aad139a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:58.353849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0ab7510a-38f1-4a9b-8017-dde85aad139a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:58.432937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7510a-38f1-4a9b-8017-dde85aad139a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:58.437918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7872daf-d507-4ccf-9809-2f9f33b596da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:58.438573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7872daf-d507-4ccf-9809-2f9f33b596da service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:58.438838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7872daf-d507-4ccf-9809-2f9f33b596da service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:58.439111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7872daf-d507-4ccf-9809-2f9f33b596da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:58.459748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7872daf-d507-4ccf-9809-2f9f33b596da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:58.461436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7872daf-d507-4ccf-9809-2f9f33b596da service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:58.473818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1111/2209] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:30:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:30:58.618090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:58.658564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:58.770635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcbaff-fe1e-4ad6-a835-001bfe2bf806 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:58.773110 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b5bcbaff-fe1e-4ad6-a835-001bfe2bf806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:58.885471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcbaff-fe1e-4ad6-a835-001bfe2bf806 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:58.890971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:58.891834 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:58.893953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:58.893953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] PATH_INFO: `/v3/projects/6e204c0f072a40fb8d7802a755681c06` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:58.893953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e204c0f072a40fb8d7802a755681c06)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:58.896063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:58.967421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f0627-63fb-4e96-bb80-d84bba07d2de None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:58.967746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f0627-63fb-4e96-bb80-d84bba07d2de None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:58.976373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f0627-63fb-4e96-bb80-d84bba07d2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:59.018034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-122f0627-63fb-4e96-bb80-d84bba07d2de None None] MFA Rules not processed for user `e7a29212dfae4a93beea7d11085cde63`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:30:59.019385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-122f0627-63fb-4e96-bb80-d84bba07d2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:59.070007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e204c0f072a40fb8d7802a755681c06', 'request_id': 'req-f1ce0efe-9904-46b6-9dad-8e350fbb6393'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:59.078491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e204c0f072a40fb8d7802a755681c06', 'request_id': 'req-f1ce0efe-9904-46b6-9dad-8e350fbb6393'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:59.103964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:30:59.142242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1112/2210] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:30:59.147656 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f0627-63fb-4e96-bb80-d84bba07d2de None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:59.149731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-122f0627-63fb-4e96-bb80-d84bba07d2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:59.166445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f0627-63fb-4e96-bb80-d84bba07d2de None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:59.169284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e204c0f072a40fb8d7802a755681c06', 'request_id': 'req-f1ce0efe-9904-46b6-9dad-8e350fbb6393'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:59.210624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] The token cache is being invalidate because project 6e204c0f072a40fb8d7802a755681c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:30:59.210921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:59.211632 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:59.213416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1099/2211] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:30:58 2022] DELETE /identity/v3/projects/6e204c0f072a40fb8d7802a755681c06 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:30:59.217182 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:59.218690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:59.240719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5380d-9f43-4c59-ab3a-6cf30e676448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:59.241413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5380d-9f43-4c59-ab3a-6cf30e676448 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:59.241696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5380d-9f43-4c59-ab3a-6cf30e676448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:59.241971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5380d-9f43-4c59-ab3a-6cf30e676448 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:59.242611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5380d-9f43-4c59-ab3a-6cf30e676448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:59.244758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5380d-9f43-4c59-ab3a-6cf30e676448 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:59.290192 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce0efe-9904-46b6-9dad-8e350fbb6393 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:59.295379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:59.296087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:59.296369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:59.296663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] PATH_INFO: `/v3/users/9c169b1feead4940857ddcdb5cbb8395` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:59.297832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c169b1feead4940857ddcdb5cbb8395)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:59.299815 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:59.352901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1113/2212] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:30:59 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:59.357017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5380d-9f43-4c59-ab3a-6cf30e676448 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:59.358778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8cf5380d-9f43-4c59-ab3a-6cf30e676448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:59.436454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5380d-9f43-4c59-ab3a-6cf30e676448 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:59.436454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf999fad-1d43-43c3-b667-5621f3557f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:59.437055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf999fad-1d43-43c3-b667-5621f3557f34 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:59.437055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf999fad-1d43-43c3-b667-5621f3557f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:59.437194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf999fad-1d43-43c3-b667-5621f3557f34 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:59.437731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf999fad-1d43-43c3-b667-5621f3557f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:59.439705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf999fad-1d43-43c3-b667-5621f3557f34 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:59.565204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1114/2213] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:30:59 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:30:59.583679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf999fad-1d43-43c3-b667-5621f3557f34 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:59.586543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf999fad-1d43-43c3-b667-5621f3557f34 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:59.591670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251e657-81bf-4e82-b44f-c47a3663bf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:59.592962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251e657-81bf-4e82-b44f-c47a3663bf89 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:59.593313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251e657-81bf-4e82-b44f-c47a3663bf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:59.593588 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251e657-81bf-4e82-b44f-c47a3663bf89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:59.594464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251e657-81bf-4e82-b44f-c47a3663bf89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:59.596740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251e657-81bf-4e82-b44f-c47a3663bf89 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:59.615210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1115/2214] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:30:59.620103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251e657-81bf-4e82-b44f-c47a3663bf89 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:59.654021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251e657-81bf-4e82-b44f-c47a3663bf89 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:30:59.660296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c169b1feead4940857ddcdb5cbb8395'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:59.668277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c169b1feead4940857ddcdb5cbb8395'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:59.674705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaefbbc-5de5-4c84-bb18-1936a4e8587f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:30:59.678874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefbbc-5de5-4c84-bb18-1936a4e8587f admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:30:59.678874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefbbc-5de5-4c84-bb18-1936a4e8587f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:30:59.678874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefbbc-5de5-4c84-bb18-1936a4e8587f admin admin] PATH_INFO: `/v3/projects/ffc9db130a0043e98bc9e518812757b4/users/2e7a483ed0c84cc699dffdc4fc8fae08/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:30:59.725817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaefbbc-5de5-4c84-bb18-1936a4e8587f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc9db130a0043e98bc9e518812757b4, user_id=2e7a483ed0c84cc699dffdc4fc8fae08, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:30:59.731312 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaefbbc-5de5-4c84-bb18-1936a4e8587f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:30:59.769203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c169b1feead4940857ddcdb5cbb8395'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:59.796191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c169b1feead4940857ddcdb5cbb8395'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:59.797579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c169b1feead4940857ddcdb5cbb8395'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:30:59.812774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1100/2215] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:30:59 2022] DELETE /identity/v3/users/9c169b1feead4940857ddcdb5cbb8395 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:59.834578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1116/2216] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:30:59 2022] PUT /identity/v3/projects/ffc9db130a0043e98bc9e518812757b4/users/2e7a483ed0c84cc699dffdc4fc8fae08/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:30:59.836426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:30:59.843086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:30:59.984231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a33d5c-c941-4620-82d6-96fa32ea827d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:00.059515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3c8e2-ab2e-4251-9f40-c591b2d4c3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:00.059515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3c8e2-ab2e-4251-9f40-c591b2d4c3dd admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:00.059515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3c8e2-ab2e-4251-9f40-c591b2d4c3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:00.059515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3c8e2-ab2e-4251-9f40-c591b2d4c3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:00.060172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3c8e2-ab2e-4251-9f40-c591b2d4c3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:00.061956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3c8e2-ab2e-4251-9f40-c591b2d4c3dd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:00.078629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1101/2217] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:00.080082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaefbbc-5de5-4c84-bb18-1936a4e8587f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:00.093048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaefbbc-5de5-4c84-bb18-1936a4e8587f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:00.106763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800bcfa-cbd8-4ed1-a45b-451c7a24ca1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:00.106763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800bcfa-cbd8-4ed1-a45b-451c7a24ca1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:00.106763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800bcfa-cbd8-4ed1-a45b-451c7a24ca1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:00.106763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800bcfa-cbd8-4ed1-a45b-451c7a24ca1f admin admin] PATH_INFO: `/v3/projects/ffc9db130a0043e98bc9e518812757b4/users/2e7a483ed0c84cc699dffdc4fc8fae08/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:00.108054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800bcfa-cbd8-4ed1-a45b-451c7a24ca1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc9db130a0043e98bc9e518812757b4, user_id=2e7a483ed0c84cc699dffdc4fc8fae08, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:00.111935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800bcfa-cbd8-4ed1-a45b-451c7a24ca1f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:00.126235 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1117/2218] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:00 2022] PUT /identity/v3/projects/ffc9db130a0043e98bc9e518812757b4/users/2e7a483ed0c84cc699dffdc4fc8fae08/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:00.233644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:00.689971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3c8e2-ab2e-4251-9f40-c591b2d4c3dd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:00.692614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6fc3c8e2-ab2e-4251-9f40-c591b2d4c3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:00.723190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3c8e2-ab2e-4251-9f40-c591b2d4c3dd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:00.792774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:00.828627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:00.829871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:00.830358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:00.830820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] PATH_INFO: `/v3/projects/91952a63ccc047208d0734ef0e10c645` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:00.833461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] RBAC: Authorizing `identity:delete_project(project_id=91952a63ccc047208d0734ef0e10c645)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:00.837177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:00.955364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91952a63ccc047208d0734ef0e10c645', 'request_id': 'req-b9ca7806-43ad-4f57-8b82-50a54de2c37c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:00.965586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91952a63ccc047208d0734ef0e10c645', 'request_id': 'req-b9ca7806-43ad-4f57-8b82-50a54de2c37c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:01.097207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91952a63ccc047208d0734ef0e10c645', 'request_id': 'req-b9ca7806-43ad-4f57-8b82-50a54de2c37c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:01.110734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] The token cache is being invalidate because project 91952a63ccc047208d0734ef0e10c645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:01.111103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:01.111952 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-b9ca7806-43ad-4f57-8b82-50a54de2c37c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:01.113551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1102/2219] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:00 2022] DELETE /identity/v3/projects/91952a63ccc047208d0734ef0e10c645 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:01.118517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0ed69-ec4c-4c4f-8023-9a9fba466a29 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:01.118802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0ed69-ec4c-4c4f-8023-9a9fba466a29 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:01.119077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0ed69-ec4c-4c4f-8023-9a9fba466a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:01.141405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-26a0ed69-ec4c-4c4f-8023-9a9fba466a29 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:01.142628 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-26a0ed69-ec4c-4c4f-8023-9a9fba466a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:01.260867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1118/2220] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:01.262870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb804a-5b4e-4270-9425-080aa9f4d836 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:01.263158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb804a-5b4e-4270-9425-080aa9f4d836 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:01.263434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb804a-5b4e-4270-9425-080aa9f4d836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:01.296493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f4bb804a-5b4e-4270-9425-080aa9f4d836 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:01.297781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f4bb804a-5b4e-4270-9425-080aa9f4d836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:01.309731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1103/2221] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:01.314064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a0ed69-ec4c-4c4f-8023-9a9fba466a29 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:01.326577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a0ed69-ec4c-4c4f-8023-9a9fba466a29 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:01.332003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab16cc3-7867-4ef8-824d-5712e065e72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:01.334389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab16cc3-7867-4ef8-824d-5712e065e72f admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:01.334851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab16cc3-7867-4ef8-824d-5712e065e72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:01.335305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab16cc3-7867-4ef8-824d-5712e065e72f admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:01.336118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab16cc3-7867-4ef8-824d-5712e065e72f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:01.336566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab16cc3-7867-4ef8-824d-5712e065e72f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:01.338373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab16cc3-7867-4ef8-824d-5712e065e72f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:01.361784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1119/2222] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:31:01.364473 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb804a-5b4e-4270-9425-080aa9f4d836 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:01.383154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb804a-5b4e-4270-9425-080aa9f4d836 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:01.391489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e02f5-63ee-4af8-a8f6-a759c1622779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:01.392590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e02f5-63ee-4af8-a8f6-a759c1622779 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:01.393086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e02f5-63ee-4af8-a8f6-a759c1622779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:01.393697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e02f5-63ee-4af8-a8f6-a759c1622779 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:01.394914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e02f5-63ee-4af8-a8f6-a759c1622779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:01.399212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e02f5-63ee-4af8-a8f6-a759c1622779 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:01.424851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:01.553007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1104/2223] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:01 2022] POST /identity/v3/projects => generated 363 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:01.554385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab16cc3-7867-4ef8-824d-5712e065e72f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:01.568183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab16cc3-7867-4ef8-824d-5712e065e72f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:01.686233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6fa60-64f5-4e16-8820-7034c9ae4f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:01.686868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6fa60-64f5-4e16-8820-7034c9ae4f59 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:01.687152 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6fa60-64f5-4e16-8820-7034c9ae4f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:01.687427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6fa60-64f5-4e16-8820-7034c9ae4f59 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:01.688163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6fa60-64f5-4e16-8820-7034c9ae4f59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:01.690854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6fa60-64f5-4e16-8820-7034c9ae4f59 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:01.809649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1120/2224] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:01 2022] POST /identity/v3/users => generated 282 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:01.810980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e02f5-63ee-4af8-a8f6-a759c1622779 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:01.823676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e02f5-63ee-4af8-a8f6-a759c1622779 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:01.833926 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce37fcd-d446-4afe-8b4e-8761e885c087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:01.834874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce37fcd-d446-4afe-8b4e-8761e885c087 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:01.835736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce37fcd-d446-4afe-8b4e-8761e885c087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:01.836561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce37fcd-d446-4afe-8b4e-8761e885c087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:01.837645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce37fcd-d446-4afe-8b4e-8761e885c087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:01.839654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce37fcd-d446-4afe-8b4e-8761e885c087 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:01.859806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1105/2225] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:01.861671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6fa60-64f5-4e16-8820-7034c9ae4f59 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:01.876444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6fa60-64f5-4e16-8820-7034c9ae4f59 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:01.898458 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:01.903871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ad557-72b6-4ab4-ace4-c51953da59f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:01.904854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ad557-72b6-4ab4-ace4-c51953da59f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:01.906218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ad557-72b6-4ab4-ace4-c51953da59f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:01.907582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ad557-72b6-4ab4-ace4-c51953da59f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:01.908478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ad557-72b6-4ab4-ace4-c51953da59f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:01.914739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ad557-72b6-4ab4-ace4-c51953da59f2 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:01.936788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce37fcd-d446-4afe-8b4e-8761e885c087 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:01.949199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1121/2226] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:01.952074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce37fcd-d446-4afe-8b4e-8761e885c087 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:01.957730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f122403-6265-4e4b-95ab-35fd6344618b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:01.959497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f122403-6265-4e4b-95ab-35fd6344618b admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:01.960015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f122403-6265-4e4b-95ab-35fd6344618b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:01.960480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f122403-6265-4e4b-95ab-35fd6344618b admin admin] PATH_INFO: `/v3/projects/8780c7a24a754f3388bf11cd78059c5f/users/64a70f8c3ac84cdbbb86aec68f01eee2/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:01.985428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f122403-6265-4e4b-95ab-35fd6344618b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8780c7a24a754f3388bf11cd78059c5f, user_id=64a70f8c3ac84cdbbb86aec68f01eee2, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:01.989689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f122403-6265-4e4b-95ab-35fd6344618b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:02.158736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1106/2227] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:01 2022] PUT /identity/v3/projects/8780c7a24a754f3388bf11cd78059c5f/users/64a70f8c3ac84cdbbb86aec68f01eee2/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:02.171428 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be390b-964d-4cab-b80c-ca8562e99d76 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:02.171980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be390b-964d-4cab-b80c-ca8562e99d76 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:02.172439 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be390b-964d-4cab-b80c-ca8562e99d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:02.203202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c7be390b-964d-4cab-b80c-ca8562e99d76 None None] MFA Rules not processed for user `64a70f8c3ac84cdbbb86aec68f01eee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:02.204828 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c7be390b-964d-4cab-b80c-ca8562e99d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:02.306778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1122/2228] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:02.311126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f122403-6265-4e4b-95ab-35fd6344618b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:02.432149 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f122403-6265-4e4b-95ab-35fd6344618b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:02.434354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d458d-f456-4469-9e77-84671c6e7926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:02.435179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d458d-f456-4469-9e77-84671c6e7926 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:02.435671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d458d-f456-4469-9e77-84671c6e7926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:02.436093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d458d-f456-4469-9e77-84671c6e7926 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:02.436887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d458d-f456-4469-9e77-84671c6e7926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:02.439197 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d458d-f456-4469-9e77-84671c6e7926 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:02.548352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1107/2229] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:02 2022] POST /identity/v3/projects => generated 365 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:02.549675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be390b-964d-4cab-b80c-ca8562e99d76 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:02.562338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be390b-964d-4cab-b80c-ca8562e99d76 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:02.654275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0954db-b2b5-4d9c-9652-e818c4957d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:02.654973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0954db-b2b5-4d9c-9652-e818c4957d51 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:02.655236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0954db-b2b5-4d9c-9652-e818c4957d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:02.655654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0954db-b2b5-4d9c-9652-e818c4957d51 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:02.658459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0954db-b2b5-4d9c-9652-e818c4957d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:02.660685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0954db-b2b5-4d9c-9652-e818c4957d51 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:02.681088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:02.737113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1123/2230] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:02 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:02.741727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d458d-f456-4469-9e77-84671c6e7926 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:02.763139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d458d-f456-4469-9e77-84671c6e7926 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:02.768325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3df76-0ddf-48c0-ac15-e380452ae73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:02.769671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3df76-0ddf-48c0-ac15-e380452ae73e admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:02.769671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3df76-0ddf-48c0-ac15-e380452ae73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:02.769671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3df76-0ddf-48c0-ac15-e380452ae73e admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:02.770175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3df76-0ddf-48c0-ac15-e380452ae73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:02.771984 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3df76-0ddf-48c0-ac15-e380452ae73e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:02.790812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1108/2231] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:02.804246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0954db-b2b5-4d9c-9652-e818c4957d51 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:02.820083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0954db-b2b5-4d9c-9652-e818c4957d51 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:02.829206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6550f-f86b-4ff5-833c-66cb7c0b93a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:02.833870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6550f-f86b-4ff5-833c-66cb7c0b93a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:02.833870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6550f-f86b-4ff5-833c-66cb7c0b93a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:02.833870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6550f-f86b-4ff5-833c-66cb7c0b93a7 admin admin] PATH_INFO: `/v3/projects/282a19ba03be42ae9ebae177190d7b51/users/0d77813978a94c66a94acc6f860ca4c8/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:02.860370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6550f-f86b-4ff5-833c-66cb7c0b93a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=282a19ba03be42ae9ebae177190d7b51, user_id=0d77813978a94c66a94acc6f860ca4c8, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:02.869792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6550f-f86b-4ff5-833c-66cb7c0b93a7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:02.958865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1124/2232] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:02 2022] PUT /identity/v3/projects/282a19ba03be42ae9ebae177190d7b51/users/0d77813978a94c66a94acc6f860ca4c8/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:02.962879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3df76-0ddf-48c0-ac15-e380452ae73e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:02.979347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3df76-0ddf-48c0-ac15-e380452ae73e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:03.104995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f4dd1-973a-4d65-8d61-b60d39d2fae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:03.106010 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f4dd1-973a-4d65-8d61-b60d39d2fae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:03.106496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f4dd1-973a-4d65-8d61-b60d39d2fae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:03.107655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f4dd1-973a-4d65-8d61-b60d39d2fae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:03.108573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f4dd1-973a-4d65-8d61-b60d39d2fae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:03.111315 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f4dd1-973a-4d65-8d61-b60d39d2fae0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:03.134522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6550f-f86b-4ff5-833c-66cb7c0b93a7 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:03.135340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1109/2233] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:03.148161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6550f-f86b-4ff5-833c-66cb7c0b93a7 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:03.154393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29859a33-67f3-4b38-a27b-a1597a0edf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:03.155483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29859a33-67f3-4b38-a27b-a1597a0edf0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:03.155982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29859a33-67f3-4b38-a27b-a1597a0edf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:03.156581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29859a33-67f3-4b38-a27b-a1597a0edf0d admin admin] PATH_INFO: `/v3/projects/282a19ba03be42ae9ebae177190d7b51/users/0d77813978a94c66a94acc6f860ca4c8/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:03.173280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29859a33-67f3-4b38-a27b-a1597a0edf0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=282a19ba03be42ae9ebae177190d7b51, user_id=0d77813978a94c66a94acc6f860ca4c8, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:03.176860 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29859a33-67f3-4b38-a27b-a1597a0edf0d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:03.190911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1125/2234] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:03 2022] PUT /identity/v3/projects/282a19ba03be42ae9ebae177190d7b51/users/0d77813978a94c66a94acc6f860ca4c8/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:03.201569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab7e77-54e7-4cbf-8962-51354311e0e3 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:03.201838 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab7e77-54e7-4cbf-8962-51354311e0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:03.202118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab7e77-54e7-4cbf-8962-51354311e0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:03.261332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c7ab7e77-54e7-4cbf-8962-51354311e0e3 None None] MFA Rules not processed for user `0d77813978a94c66a94acc6f860ca4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:03.266611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c7ab7e77-54e7-4cbf-8962-51354311e0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:03.377645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:03.397882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1110/2235] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:03.410108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29859a33-67f3-4b38-a27b-a1597a0edf0d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:03.414508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-29859a33-67f3-4b38-a27b-a1597a0edf0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:03.470214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:03.522057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29859a33-67f3-4b38-a27b-a1597a0edf0d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:03.534993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723f57d-2030-47bc-8b01-e4564f940cdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:03.535300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723f57d-2030-47bc-8b01-e4564f940cdc service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:03.535857 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723f57d-2030-47bc-8b01-e4564f940cdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:03.536660 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723f57d-2030-47bc-8b01-e4564f940cdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:03.553423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723f57d-2030-47bc-8b01-e4564f940cdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:03.555140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723f57d-2030-47bc-8b01-e4564f940cdc service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:03.566474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1126/2236] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:03.827594 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab7e77-54e7-4cbf-8962-51354311e0e3 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:03.843379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab7e77-54e7-4cbf-8962-51354311e0e3 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:03.847334 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b8416-abe5-47c3-a79b-7e5cf615129c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:03.847993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b8416-abe5-47c3-a79b-7e5cf615129c service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:03.848269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b8416-abe5-47c3-a79b-7e5cf615129c service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:03.849196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b8416-abe5-47c3-a79b-7e5cf615129c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:03.964205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b8416-abe5-47c3-a79b-7e5cf615129c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:03.971382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b8416-abe5-47c3-a79b-7e5cf615129c service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:03.995770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1111/2237] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:04.463464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:05.115130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e01c22-47f9-4ed7-9043-33e18da71d81 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:05.115130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e01c22-47f9-4ed7-9043-33e18da71d81 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:05.115130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e01c22-47f9-4ed7-9043-33e18da71d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:05.134309 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-02e01c22-47f9-4ed7-9043-33e18da71d81 None None] MFA Rules not processed for user `a669f6cbac5a4d84a9e71c27c01969ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:05.135257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:05.135773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-02e01c22-47f9-4ed7-9043-33e18da71d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:05.192671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1127/2238] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:06.012579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:06.084614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:07.572395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:07.734864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04c0a7-593b-40a1-9f38-b741952b493c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:07.735862 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04c0a7-593b-40a1-9f38-b741952b493c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:07.736406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04c0a7-593b-40a1-9f38-b741952b493c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:07.777308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3d04c0a7-593b-40a1-9f38-b741952b493c None None] MFA Rules not processed for user `2e7a483ed0c84cc699dffdc4fc8fae08`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:07.778727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3d04c0a7-593b-40a1-9f38-b741952b493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:07.789799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:07.803572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:07.861629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1112/2239] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:07.876321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e01c22-47f9-4ed7-9043-33e18da71d81 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:07.877749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-02e01c22-47f9-4ed7-9043-33e18da71d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:07.981243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e01c22-47f9-4ed7-9043-33e18da71d81 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:07.987002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5646b-6839-471e-af5b-5b109be74f74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:07.987663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5646b-6839-471e-af5b-5b109be74f74 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:07.987949 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5646b-6839-471e-af5b-5b109be74f74 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:07.988226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5646b-6839-471e-af5b-5b109be74f74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:07.990029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-93c5646b-6839-471e-af5b-5b109be74f74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:08.018120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:08.054299 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5646b-6839-471e-af5b-5b109be74f74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:08.056087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5646b-6839-471e-af5b-5b109be74f74 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:08.066176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1128/2240] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:08.114208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:10.389940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04c0a7-593b-40a1-9f38-b741952b493c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:10.492727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04c0a7-593b-40a1-9f38-b741952b493c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:10.503665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:10.504702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:10.505131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:10.505618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] PATH_INFO: `/v3/users/64a70f8c3ac84cdbbb86aec68f01eee2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:10.507176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a70f8c3ac84cdbbb86aec68f01eee2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:10.510320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:10.869038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a70f8c3ac84cdbbb86aec68f01eee2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:10.869697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a70f8c3ac84cdbbb86aec68f01eee2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:11.003479 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a70f8c3ac84cdbbb86aec68f01eee2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:11.020154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a70f8c3ac84cdbbb86aec68f01eee2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:11.024135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a70f8c3ac84cdbbb86aec68f01eee2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:11.045840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1113/2241] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:10 2022] DELETE /identity/v3/users/64a70f8c3ac84cdbbb86aec68f01eee2 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:11.053350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911ef12-beca-4e08-a290-a95a9e22ae2c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:11.053899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911ef12-beca-4e08-a290-a95a9e22ae2c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:11.053899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911ef12-beca-4e08-a290-a95a9e22ae2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:11.084145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-b911ef12-beca-4e08-a290-a95a9e22ae2c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:11.085693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b911ef12-beca-4e08-a290-a95a9e22ae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:11.159818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1129/2242] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:11.174582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:11.176057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:11.237619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:11.260905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a26ff-5118-44ae-a48a-d2b519ed394f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:11.266715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707459b9-2aa9-460f-893a-7f431335930d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:11.266715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707459b9-2aa9-460f-893a-7f431335930d service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:11.266973 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707459b9-2aa9-460f-893a-7f431335930d service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:11.267222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707459b9-2aa9-460f-893a-7f431335930d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:11.280658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707459b9-2aa9-460f-893a-7f431335930d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:11.282415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707459b9-2aa9-460f-893a-7f431335930d service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:11.296296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1114/2243] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:12.054501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911ef12-beca-4e08-a290-a95a9e22ae2c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:12.057525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b911ef12-beca-4e08-a290-a95a9e22ae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:12.087071 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911ef12-beca-4e08-a290-a95a9e22ae2c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:12.103886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:12.105304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:12.105798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:12.107872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] PATH_INFO: `/v3/projects/8780c7a24a754f3388bf11cd78059c5f` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:12.112823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8780c7a24a754f3388bf11cd78059c5f)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:12.115979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:12.142195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:12.327093 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8780c7a24a754f3388bf11cd78059c5f', 'request_id': 'req-8122f89c-a88f-41b0-ae1e-89b3d0badf65'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:12.333405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8780c7a24a754f3388bf11cd78059c5f', 'request_id': 'req-8122f89c-a88f-41b0-ae1e-89b3d0badf65'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:12.469299 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8780c7a24a754f3388bf11cd78059c5f', 'request_id': 'req-8122f89c-a88f-41b0-ae1e-89b3d0badf65'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:12.482640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] The token cache is being invalidate because project 8780c7a24a754f3388bf11cd78059c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:12.483252 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:12.484161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:12.485695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1130/2244] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:12 2022] DELETE /identity/v3/projects/8780c7a24a754f3388bf11cd78059c5f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:12.490463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707459b9-2aa9-460f-893a-7f431335930d service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:12.500640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707459b9-2aa9-460f-893a-7f431335930d service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:12.583229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:12.583901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:12.584500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:12.584500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] PATH_INFO: `/v3/users/0d77813978a94c66a94acc6f860ca4c8` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:12.585689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d77813978a94c66a94acc6f860ca4c8)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:12.587673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:12.870146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d77813978a94c66a94acc6f860ca4c8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:12.870862 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d77813978a94c66a94acc6f860ca4c8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:12.949530 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d77813978a94c66a94acc6f860ca4c8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:12.966537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d77813978a94c66a94acc6f860ca4c8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:12.967305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca9044fa-bee2-46d0-ba5e-7d6d448bea4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d77813978a94c66a94acc6f860ca4c8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:12.987225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1115/2245] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:12 2022] DELETE /identity/v3/users/0d77813978a94c66a94acc6f860ca4c8 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:13.944304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:13.947231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:13.965534 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122f89c-a88f-41b0-ae1e-89b3d0badf65 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:14.036230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:14.037298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:14.037442 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:14.037722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] PATH_INFO: `/v3/projects/282a19ba03be42ae9ebae177190d7b51` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:14.039003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=282a19ba03be42ae9ebae177190d7b51)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:14.041063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:14.270039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '282a19ba03be42ae9ebae177190d7b51', 'request_id': 'req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:14.279092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '282a19ba03be42ae9ebae177190d7b51', 'request_id': 'req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:14.437921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '282a19ba03be42ae9ebae177190d7b51', 'request_id': 'req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:14.454551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] The token cache is being invalidate because project 282a19ba03be42ae9ebae177190d7b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:14.454844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:14.455559 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a6d83b69-4194-4b61-8f7e-ed0ccbfaf899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:14.465933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1131/2246] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:13 2022] DELETE /identity/v3/projects/282a19ba03be42ae9ebae177190d7b51 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:14.467718 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf6cf-4e97-4737-9f86-b90fb7ff9f55 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:14.469102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf6cf-4e97-4737-9f86-b90fb7ff9f55 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:14.470130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf6cf-4e97-4737-9f86-b90fb7ff9f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:14.529787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-cefaf6cf-4e97-4737-9f86-b90fb7ff9f55 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:14.532142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-cefaf6cf-4e97-4737-9f86-b90fb7ff9f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:14.618469 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:14.628063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1116/2247] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:14.636324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4441f-74a6-43ce-b95b-03cb8116c247 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:14.636602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4441f-74a6-43ce-b95b-03cb8116c247 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:14.636876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4441f-74a6-43ce-b95b-03cb8116c247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:14.662426 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-22d4441f-74a6-43ce-b95b-03cb8116c247 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:14.664743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-22d4441f-74a6-43ce-b95b-03cb8116c247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:14.682565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1132/2248] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:14.684061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf6cf-4e97-4737-9f86-b90fb7ff9f55 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:14.696956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf6cf-4e97-4737-9f86-b90fb7ff9f55 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:14.703486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d010f66-36bb-4048-ab92-347b90e3e8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:14.704143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d010f66-36bb-4048-ab92-347b90e3e8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:14.708781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d010f66-36bb-4048-ab92-347b90e3e8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:14.708781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d010f66-36bb-4048-ab92-347b90e3e8e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:14.708781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d010f66-36bb-4048-ab92-347b90e3e8e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:14.708781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d010f66-36bb-4048-ab92-347b90e3e8e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:14.708781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d010f66-36bb-4048-ab92-347b90e3e8e9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:14.726387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1117/2249] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:31:14.727853 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4441f-74a6-43ce-b95b-03cb8116c247 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:14.738545 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4441f-74a6-43ce-b95b-03cb8116c247 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:14.746598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244b4c0-3a59-4deb-8d37-9e8d34c53ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:14.747675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244b4c0-3a59-4deb-8d37-9e8d34c53ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:14.748109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244b4c0-3a59-4deb-8d37-9e8d34c53ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:14.748574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244b4c0-3a59-4deb-8d37-9e8d34c53ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:14.750293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244b4c0-3a59-4deb-8d37-9e8d34c53ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:14.753670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244b4c0-3a59-4deb-8d37-9e8d34c53ae9 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:14.895818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:14.915120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1133/2250] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:14 2022] POST /identity/v3/projects => generated 369 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:14.916563 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d010f66-36bb-4048-ab92-347b90e3e8e9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:14.934889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d010f66-36bb-4048-ab92-347b90e3e8e9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:15.025478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcb6fb-392b-4142-9eaf-a760592d6ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:15.026153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcb6fb-392b-4142-9eaf-a760592d6ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:15.026551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcb6fb-392b-4142-9eaf-a760592d6ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:15.027083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcb6fb-392b-4142-9eaf-a760592d6ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:15.027778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcb6fb-392b-4142-9eaf-a760592d6ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:15.037466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcb6fb-392b-4142-9eaf-a760592d6ef3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:15.211738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1118/2251] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:14 2022] POST /identity/v3/users => generated 285 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:15.216224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244b4c0-3a59-4deb-8d37-9e8d34c53ae9 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:15.233305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244b4c0-3a59-4deb-8d37-9e8d34c53ae9 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:15.238183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626183fa-fc0d-44eb-bf4c-00b77ffca696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:15.238810 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626183fa-fc0d-44eb-bf4c-00b77ffca696 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:15.239095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626183fa-fc0d-44eb-bf4c-00b77ffca696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:15.239370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626183fa-fc0d-44eb-bf4c-00b77ffca696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:15.240554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626183fa-fc0d-44eb-bf4c-00b77ffca696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:15.242192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626183fa-fc0d-44eb-bf4c-00b77ffca696 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:15.252883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1134/2252] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:15.257599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcb6fb-392b-4142-9eaf-a760592d6ef3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:15.268155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcb6fb-392b-4142-9eaf-a760592d6ef3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:15.272893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fbd9c-001e-456d-8db3-729042534742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:15.273507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fbd9c-001e-456d-8db3-729042534742 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:15.273783 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fbd9c-001e-456d-8db3-729042534742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:15.274055 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fbd9c-001e-456d-8db3-729042534742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:15.274700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fbd9c-001e-456d-8db3-729042534742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:15.276451 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fbd9c-001e-456d-8db3-729042534742 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:15.288264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1119/2253] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:15.293045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626183fa-fc0d-44eb-bf4c-00b77ffca696 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:15.305719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626183fa-fc0d-44eb-bf4c-00b77ffca696 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:15.310670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae11ff5-7cb6-4abb-bb59-3377a5c1c2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:15.311378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae11ff5-7cb6-4abb-bb59-3377a5c1c2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:15.311715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae11ff5-7cb6-4abb-bb59-3377a5c1c2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:15.311965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae11ff5-7cb6-4abb-bb59-3377a5c1c2e1 admin admin] PATH_INFO: `/v3/projects/c98d0a4350eb4db1a1b45a26a6b8b928/users/780b1b79340c4ae69b0f679829c4e436/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:15.329582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae11ff5-7cb6-4abb-bb59-3377a5c1c2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c98d0a4350eb4db1a1b45a26a6b8b928, user_id=780b1b79340c4ae69b0f679829c4e436, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:15.332395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae11ff5-7cb6-4abb-bb59-3377a5c1c2e1 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:15.469705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1135/2254] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:15 2022] PUT /identity/v3/projects/c98d0a4350eb4db1a1b45a26a6b8b928/users/780b1b79340c4ae69b0f679829c4e436/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:15.482516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520cfb0-8159-4652-9515-5c77fe83e719 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:15.483056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520cfb0-8159-4652-9515-5c77fe83e719 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:15.483150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520cfb0-8159-4652-9515-5c77fe83e719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:15.519258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4520cfb0-8159-4652-9515-5c77fe83e719 None None] MFA Rules not processed for user `780b1b79340c4ae69b0f679829c4e436`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:15.521112 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4520cfb0-8159-4652-9515-5c77fe83e719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:15.614097 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1120/2255] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:15.619415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae11ff5-7cb6-4abb-bb59-3377a5c1c2e1 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:15.702148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae11ff5-7cb6-4abb-bb59-3377a5c1c2e1 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:15.707507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8c804-1118-46c9-a965-cb59c6269194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:15.708170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8c804-1118-46c9-a965-cb59c6269194 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:15.708452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8c804-1118-46c9-a965-cb59c6269194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:15.709245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8c804-1118-46c9-a965-cb59c6269194 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:15.709896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd8c804-1118-46c9-a965-cb59c6269194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:15.711837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd8c804-1118-46c9-a965-cb59c6269194 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:15.865210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1136/2256] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:15 2022] POST /identity/v3/projects => generated 367 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:15.866682 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520cfb0-8159-4652-9515-5c77fe83e719 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:15.881478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520cfb0-8159-4652-9515-5c77fe83e719 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:15.966107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7a47e-1f3a-47b5-b09d-80271bea6a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:15.967842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7a47e-1f3a-47b5-b09d-80271bea6a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:15.967842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7a47e-1f3a-47b5-b09d-80271bea6a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:15.967842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7a47e-1f3a-47b5-b09d-80271bea6a64 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:15.968124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7a47e-1f3a-47b5-b09d-80271bea6a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:15.970034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7a47e-1f3a-47b5-b09d-80271bea6a64 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:16.130070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1121/2257] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:15 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:16.134416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8c804-1118-46c9-a965-cb59c6269194 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:16.145242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8c804-1118-46c9-a965-cb59c6269194 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:16.150669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b22bfc-cb00-4117-82d0-581f606959e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:16.151298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b22bfc-cb00-4117-82d0-581f606959e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:16.151602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b22bfc-cb00-4117-82d0-581f606959e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:16.151876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b22bfc-cb00-4117-82d0-581f606959e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:16.152805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b22bfc-cb00-4117-82d0-581f606959e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:16.154303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b22bfc-cb00-4117-82d0-581f606959e3 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:16.168863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1137/2258] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:16.170460 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7a47e-1f3a-47b5-b09d-80271bea6a64 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:16.184142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7a47e-1f3a-47b5-b09d-80271bea6a64 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:16.189345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bc08e-bb41-4489-b54e-b5357933b039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:16.191861 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bc08e-bb41-4489-b54e-b5357933b039 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:16.191861 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bc08e-bb41-4489-b54e-b5357933b039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:16.191861 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bc08e-bb41-4489-b54e-b5357933b039 admin admin] PATH_INFO: `/v3/projects/296b2791fccc4365807381dc302c01a6/users/77d0284e2a4143d1ba31842947972386/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:16.210471 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bc08e-bb41-4489-b54e-b5357933b039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296b2791fccc4365807381dc302c01a6, user_id=77d0284e2a4143d1ba31842947972386, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:16.213323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bc08e-bb41-4489-b54e-b5357933b039 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:16.306793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1122/2259] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:16 2022] PUT /identity/v3/projects/296b2791fccc4365807381dc302c01a6/users/77d0284e2a4143d1ba31842947972386/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:16.310814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b22bfc-cb00-4117-82d0-581f606959e3 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:16.321868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b22bfc-cb00-4117-82d0-581f606959e3 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:16.383945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a39d9-7681-4891-94e0-a1302f022e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:16.384604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a39d9-7681-4891-94e0-a1302f022e58 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:16.384887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a39d9-7681-4891-94e0-a1302f022e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:16.385161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a39d9-7681-4891-94e0-a1302f022e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:16.385801 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a39d9-7681-4891-94e0-a1302f022e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:16.387492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a39d9-7681-4891-94e0-a1302f022e58 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:16.398988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1138/2260] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:16.403277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bc08e-bb41-4489-b54e-b5357933b039 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:16.413941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bc08e-bb41-4489-b54e-b5357933b039 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:16.418738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5ad87-f8ec-41f9-8a09-3e167be79bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:16.419396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5ad87-f8ec-41f9-8a09-3e167be79bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:16.419717 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5ad87-f8ec-41f9-8a09-3e167be79bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:16.419996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5ad87-f8ec-41f9-8a09-3e167be79bbd admin admin] PATH_INFO: `/v3/projects/296b2791fccc4365807381dc302c01a6/users/77d0284e2a4143d1ba31842947972386/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:16.422122 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5ad87-f8ec-41f9-8a09-3e167be79bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=296b2791fccc4365807381dc302c01a6, user_id=77d0284e2a4143d1ba31842947972386, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:16.425129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5ad87-f8ec-41f9-8a09-3e167be79bbd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:16.444706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1123/2261] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:16 2022] PUT /identity/v3/projects/296b2791fccc4365807381dc302c01a6/users/77d0284e2a4143d1ba31842947972386/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:16.455330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3e21-bb9d-488a-8cfd-f5264bfd409c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:16.455672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3e21-bb9d-488a-8cfd-f5264bfd409c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:16.455952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3e21-bb9d-488a-8cfd-f5264bfd409c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:16.486171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-70fb3e21-bb9d-488a-8cfd-f5264bfd409c None None] MFA Rules not processed for user `77d0284e2a4143d1ba31842947972386`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:16.487503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-70fb3e21-bb9d-488a-8cfd-f5264bfd409c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:16.571364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1139/2262] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:16.585931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5ad87-f8ec-41f9-8a09-3e167be79bbd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:16.588195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-31e5ad87-f8ec-41f9-8a09-3e167be79bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:16.650648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5ad87-f8ec-41f9-8a09-3e167be79bbd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:16.654828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4f074-2f28-4f21-8be3-3fe35d9d18db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:16.655439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4f074-2f28-4f21-8be3-3fe35d9d18db service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:16.655742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4f074-2f28-4f21-8be3-3fe35d9d18db service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:16.656015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4f074-2f28-4f21-8be3-3fe35d9d18db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:16.665777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4f074-2f28-4f21-8be3-3fe35d9d18db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:16.667218 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4f074-2f28-4f21-8be3-3fe35d9d18db service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:16.678364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1124/2263] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:16.841135 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb3e21-bb9d-488a-8cfd-f5264bfd409c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:16.856981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb3e21-bb9d-488a-8cfd-f5264bfd409c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:16.863243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d93a56-91a7-4f78-bd16-18f79a092b45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:16.863886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d93a56-91a7-4f78-bd16-18f79a092b45 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:16.864161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d93a56-91a7-4f78-bd16-18f79a092b45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:16.864429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d93a56-91a7-4f78-bd16-18f79a092b45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:16.926028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d93a56-91a7-4f78-bd16-18f79a092b45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:16.927091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d93a56-91a7-4f78-bd16-18f79a092b45 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:16.937950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1140/2264] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:16.966877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:17.410669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:22.700864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4f074-2f28-4f21-8be3-3fe35d9d18db service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:22.765625 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4f074-2f28-4f21-8be3-3fe35d9d18db service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:22.770458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:22.771637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:22.771942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:22.772245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] PATH_INFO: `/v3/users/780b1b79340c4ae69b0f679829c4e436` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:22.773490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=780b1b79340c4ae69b0f679829c4e436)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:22.775405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:22.995323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '780b1b79340c4ae69b0f679829c4e436'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:22.995947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '780b1b79340c4ae69b0f679829c4e436'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:22.998688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:23.206684 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '780b1b79340c4ae69b0f679829c4e436'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:23.221367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '780b1b79340c4ae69b0f679829c4e436'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:23.222160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '780b1b79340c4ae69b0f679829c4e436'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:23.236643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1125/2265] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:22 2022] DELETE /identity/v3/users/780b1b79340c4ae69b0f679829c4e436 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:23.241136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e528e4-1581-47ae-9d4a-a1aa7c85821c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:23.241951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e528e4-1581-47ae-9d4a-a1aa7c85821c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:23.242394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e528e4-1581-47ae-9d4a-a1aa7c85821c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:23.274293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d4e528e4-1581-47ae-9d4a-a1aa7c85821c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:23.275418 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d4e528e4-1581-47ae-9d4a-a1aa7c85821c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:23.343924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1141/2266] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:23.359053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:23.360609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:23.434545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf70be5-9db8-4603-bc0c-919a112eaaa9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:23.439951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7ed2a-f0c8-4ae3-87d7-62a6fde2d6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:23.440544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7ed2a-f0c8-4ae3-87d7-62a6fde2d6eb service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:23.440825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7ed2a-f0c8-4ae3-87d7-62a6fde2d6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:23.441099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7ed2a-f0c8-4ae3-87d7-62a6fde2d6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:23.452210 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7ed2a-f0c8-4ae3-87d7-62a6fde2d6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:23.453749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7ed2a-f0c8-4ae3-87d7-62a6fde2d6eb service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:23.464416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1126/2267] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:23.944400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e528e4-1581-47ae-9d4a-a1aa7c85821c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:23.946117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d4e528e4-1581-47ae-9d4a-a1aa7c85821c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:23.960643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e528e4-1581-47ae-9d4a-a1aa7c85821c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:23.966354 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:23.967311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:23.967639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:23.967957 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] PATH_INFO: `/v3/projects/c98d0a4350eb4db1a1b45a26a6b8b928` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:23.969176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c98d0a4350eb4db1a1b45a26a6b8b928)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:23.971174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:24.137044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c98d0a4350eb4db1a1b45a26a6b8b928', 'request_id': 'req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:24.140912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:24.143143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c98d0a4350eb4db1a1b45a26a6b8b928', 'request_id': 'req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:24.245306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c98d0a4350eb4db1a1b45a26a6b8b928', 'request_id': 'req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:24.259084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] The token cache is being invalidate because project c98d0a4350eb4db1a1b45a26a6b8b928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:24.259692 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:24.260432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:24.261995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1142/2268] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:23 2022] DELETE /identity/v3/projects/c98d0a4350eb4db1a1b45a26a6b8b928 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:24.276260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7ed2a-f0c8-4ae3-87d7-62a6fde2d6eb service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:24.289025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7ed2a-f0c8-4ae3-87d7-62a6fde2d6eb service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:24.355511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:24.356174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:24.356456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:24.356734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] PATH_INFO: `/v3/users/77d0284e2a4143d1ba31842947972386` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:24.360629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=77d0284e2a4143d1ba31842947972386)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:24.362615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:24.609946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77d0284e2a4143d1ba31842947972386'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:24.612385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77d0284e2a4143d1ba31842947972386'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:24.674086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77d0284e2a4143d1ba31842947972386'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:24.693750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77d0284e2a4143d1ba31842947972386'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:24.694509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c4e0c4b5-1e8a-45c2-b487-87b70dd63b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77d0284e2a4143d1ba31842947972386'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:24.696190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:24.712398 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1127/2269] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:24 2022] DELETE /identity/v3/users/77d0284e2a4143d1ba31842947972386 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:25.230199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:25.231740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:25.246734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d246df-f3d0-4b1a-b7e1-fd58323c75f3 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:25.324922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:25.325654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:25.325936 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:25.331711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] PATH_INFO: `/v3/projects/296b2791fccc4365807381dc302c01a6` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:25.331711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] RBAC: Authorizing `identity:delete_project(project_id=296b2791fccc4365807381dc302c01a6)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:25.331711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:25.474203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296b2791fccc4365807381dc302c01a6', 'request_id': 'req-c5607482-e38b-4337-84ba-09145636270f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:25.481427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296b2791fccc4365807381dc302c01a6', 'request_id': 'req-c5607482-e38b-4337-84ba-09145636270f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:25.597894 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296b2791fccc4365807381dc302c01a6', 'request_id': 'req-c5607482-e38b-4337-84ba-09145636270f'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:25.611087 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] The token cache is being invalidate because project 296b2791fccc4365807381dc302c01a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:25.611519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:25.612201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c5607482-e38b-4337-84ba-09145636270f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:25.613718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1143/2270] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:25 2022] DELETE /identity/v3/projects/296b2791fccc4365807381dc302c01a6 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:25.619193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1214e1-2997-4a22-a8ca-58998bb1ce7c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:25.619980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1214e1-2997-4a22-a8ca-58998bb1ce7c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:25.619980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1214e1-2997-4a22-a8ca-58998bb1ce7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:25.643295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7d1214e1-2997-4a22-a8ca-58998bb1ce7c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:25.644884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7d1214e1-2997-4a22-a8ca-58998bb1ce7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:25.735522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1128/2271] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:25.739466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd4d0e-f304-44bf-a401-1efa75699eed None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:25.740508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd4d0e-f304-44bf-a401-1efa75699eed None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:25.741085 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd4d0e-f304-44bf-a401-1efa75699eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:25.773031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-b0dd4d0e-f304-44bf-a401-1efa75699eed None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:25.774753 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b0dd4d0e-f304-44bf-a401-1efa75699eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:25.784297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1144/2272] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:25.788761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1214e1-2997-4a22-a8ca-58998bb1ce7c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:25.799523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1214e1-2997-4a22-a8ca-58998bb1ce7c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:25.815300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de97445-c48a-4804-9c10-ef5aab90f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:25.816744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de97445-c48a-4804-9c10-ef5aab90f0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:25.817329 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de97445-c48a-4804-9c10-ef5aab90f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:25.817654 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de97445-c48a-4804-9c10-ef5aab90f0c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:25.818974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de97445-c48a-4804-9c10-ef5aab90f0c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:25.819318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de97445-c48a-4804-9c10-ef5aab90f0c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:25.830969 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de97445-c48a-4804-9c10-ef5aab90f0c3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:25.847605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1129/2273] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:31:25.851525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd4d0e-f304-44bf-a401-1efa75699eed None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:25.931262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd4d0e-f304-44bf-a401-1efa75699eed None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:25.931262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b76833-8f63-4bfc-837a-3eb8d93c82de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:25.931262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b76833-8f63-4bfc-837a-3eb8d93c82de admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:25.931262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b76833-8f63-4bfc-837a-3eb8d93c82de admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:25.931866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b76833-8f63-4bfc-837a-3eb8d93c82de admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:25.931866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b76833-8f63-4bfc-837a-3eb8d93c82de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:25.931866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b76833-8f63-4bfc-837a-3eb8d93c82de admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:26.356607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1145/2274] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:25 2022] POST /identity/v3/projects => generated 381 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:26.357893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de97445-c48a-4804-9c10-ef5aab90f0c3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:26.387169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de97445-c48a-4804-9c10-ef5aab90f0c3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:26.478341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d25845-1a7c-4d23-88f7-b6df81911e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:26.479499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d25845-1a7c-4d23-88f7-b6df81911e10 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:26.479499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d25845-1a7c-4d23-88f7-b6df81911e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:26.479869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d25845-1a7c-4d23-88f7-b6df81911e10 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:26.480562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d25845-1a7c-4d23-88f7-b6df81911e10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:26.482553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d25845-1a7c-4d23-88f7-b6df81911e10 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:26.597137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1130/2275] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:26 2022] POST /identity/v3/users => generated 291 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:26.601612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b76833-8f63-4bfc-837a-3eb8d93c82de admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:26.614981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b76833-8f63-4bfc-837a-3eb8d93c82de admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:26.621362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceba5ae-a7a5-4b1e-9a7a-51faf911468c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:26.626629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceba5ae-a7a5-4b1e-9a7a-51faf911468c admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:26.626629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceba5ae-a7a5-4b1e-9a7a-51faf911468c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:26.626629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceba5ae-a7a5-4b1e-9a7a-51faf911468c admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:26.626629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceba5ae-a7a5-4b1e-9a7a-51faf911468c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:26.626629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceba5ae-a7a5-4b1e-9a7a-51faf911468c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:26.638501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1146/2276] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:26.642783 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d25845-1a7c-4d23-88f7-b6df81911e10 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:26.654855 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d25845-1a7c-4d23-88f7-b6df81911e10 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:26.669262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbee721-17a2-4437-93c2-303925b515f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:26.669262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbee721-17a2-4437-93c2-303925b515f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:26.669262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbee721-17a2-4437-93c2-303925b515f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:26.669262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbee721-17a2-4437-93c2-303925b515f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:26.669688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbee721-17a2-4437-93c2-303925b515f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:26.669688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbee721-17a2-4437-93c2-303925b515f7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:26.676481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1131/2277] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:26.680890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceba5ae-a7a5-4b1e-9a7a-51faf911468c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:26.691447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceba5ae-a7a5-4b1e-9a7a-51faf911468c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:26.696879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6120a9c-715d-44ab-9154-784d1b556375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:26.697545 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6120a9c-715d-44ab-9154-784d1b556375 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:26.697827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6120a9c-715d-44ab-9154-784d1b556375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:26.698101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6120a9c-715d-44ab-9154-784d1b556375 admin admin] PATH_INFO: `/v3/projects/5defa47cffff44fdb446b0cb089d2029/users/c57384ab38a3408cbde3e0db0742ebcd/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:26.716767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6120a9c-715d-44ab-9154-784d1b556375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5defa47cffff44fdb446b0cb089d2029, user_id=c57384ab38a3408cbde3e0db0742ebcd, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:26.721021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6120a9c-715d-44ab-9154-784d1b556375 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:26.801993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1147/2278] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:26 2022] PUT /identity/v3/projects/5defa47cffff44fdb446b0cb089d2029/users/c57384ab38a3408cbde3e0db0742ebcd/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:26.814122 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe7a4f-0790-4075-bdbf-57c12e5ec813 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:26.814447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe7a4f-0790-4075-bdbf-57c12e5ec813 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:26.815454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe7a4f-0790-4075-bdbf-57c12e5ec813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:26.861783 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-33fe7a4f-0790-4075-bdbf-57c12e5ec813 None None] MFA Rules not processed for user `c57384ab38a3408cbde3e0db0742ebcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:26.863286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-33fe7a4f-0790-4075-bdbf-57c12e5ec813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:26.926317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:26.949896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1132/2279] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:26.950934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6120a9c-715d-44ab-9154-784d1b556375 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:26.964987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:27.038396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6120a9c-715d-44ab-9154-784d1b556375 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:27.043170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510817c-1987-4680-a252-e9604fe9bd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:27.043859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510817c-1987-4680-a252-e9604fe9bd14 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:27.044141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510817c-1987-4680-a252-e9604fe9bd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:27.044413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510817c-1987-4680-a252-e9604fe9bd14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:27.045045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510817c-1987-4680-a252-e9604fe9bd14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:27.047199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510817c-1987-4680-a252-e9604fe9bd14 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:27.162103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1148/2280] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:26 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:27.166210 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe7a4f-0790-4075-bdbf-57c12e5ec813 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:27.178254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe7a4f-0790-4075-bdbf-57c12e5ec813 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:27.239771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadb3d0-5529-4871-a154-1de63b498679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:27.240873 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadb3d0-5529-4871-a154-1de63b498679 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:27.241149 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadb3d0-5529-4871-a154-1de63b498679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:27.241448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadb3d0-5529-4871-a154-1de63b498679 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:27.242128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eadb3d0-5529-4871-a154-1de63b498679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:27.244047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eadb3d0-5529-4871-a154-1de63b498679 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:27.351399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1133/2281] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:27 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:27.356030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510817c-1987-4680-a252-e9604fe9bd14 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:27.368773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510817c-1987-4680-a252-e9604fe9bd14 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:27.389378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e10b80-868e-4c30-82e5-1c07ab68da85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:27.389378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e10b80-868e-4c30-82e5-1c07ab68da85 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:27.389378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e10b80-868e-4c30-82e5-1c07ab68da85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:27.389378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e10b80-868e-4c30-82e5-1c07ab68da85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:27.389904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e10b80-868e-4c30-82e5-1c07ab68da85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:27.389904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e10b80-868e-4c30-82e5-1c07ab68da85 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:27.391898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1149/2282] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:27.396169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadb3d0-5529-4871-a154-1de63b498679 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:27.408314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadb3d0-5529-4871-a154-1de63b498679 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:27.413443 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd9d71-c413-4507-b578-19ebe4d7ef80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:27.414132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd9d71-c413-4507-b578-19ebe4d7ef80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:27.414481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd9d71-c413-4507-b578-19ebe4d7ef80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:27.415031 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd9d71-c413-4507-b578-19ebe4d7ef80 admin admin] PATH_INFO: `/v3/projects/5f899b7bf2b6427797fb2b4ced4c0340/users/6e4225f72fbb4a06a638bbcd08aeacc9/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:27.435310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd9d71-c413-4507-b578-19ebe4d7ef80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f899b7bf2b6427797fb2b4ced4c0340, user_id=6e4225f72fbb4a06a638bbcd08aeacc9, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:27.438139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd9d71-c413-4507-b578-19ebe4d7ef80 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:27.584624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1134/2283] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:27 2022] PUT /identity/v3/projects/5f899b7bf2b6427797fb2b4ced4c0340/users/6e4225f72fbb4a06a638bbcd08aeacc9/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:27.588975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e10b80-868e-4c30-82e5-1c07ab68da85 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:27.602291 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e10b80-868e-4c30-82e5-1c07ab68da85 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:27.665196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f919d94-76d3-48d8-bd41-4da1a7f1472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:27.665763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f919d94-76d3-48d8-bd41-4da1a7f1472b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:27.666089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f919d94-76d3-48d8-bd41-4da1a7f1472b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:27.666381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f919d94-76d3-48d8-bd41-4da1a7f1472b admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:27.667005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f919d94-76d3-48d8-bd41-4da1a7f1472b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:27.668743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f919d94-76d3-48d8-bd41-4da1a7f1472b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:27.681018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1150/2284] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:27.685537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd9d71-c413-4507-b578-19ebe4d7ef80 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:27.708924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd9d71-c413-4507-b578-19ebe4d7ef80 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:27.726738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78065594-2ed2-4130-882a-ced7176cd86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:27.735514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78065594-2ed2-4130-882a-ced7176cd86a admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:27.735514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78065594-2ed2-4130-882a-ced7176cd86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:27.735514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78065594-2ed2-4130-882a-ced7176cd86a admin admin] PATH_INFO: `/v3/projects/5f899b7bf2b6427797fb2b4ced4c0340/users/6e4225f72fbb4a06a638bbcd08aeacc9/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:27.735514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78065594-2ed2-4130-882a-ced7176cd86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f899b7bf2b6427797fb2b4ced4c0340, user_id=6e4225f72fbb4a06a638bbcd08aeacc9, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:27.735514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78065594-2ed2-4130-882a-ced7176cd86a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:27.745487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1135/2285] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:27 2022] PUT /identity/v3/projects/5f899b7bf2b6427797fb2b4ced4c0340/users/6e4225f72fbb4a06a638bbcd08aeacc9/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:27.756325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee626d-78e0-49e6-af27-48c0a3d2a092 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:27.756702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee626d-78e0-49e6-af27-48c0a3d2a092 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:27.756954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee626d-78e0-49e6-af27-48c0a3d2a092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:27.786279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-cbee626d-78e0-49e6-af27-48c0a3d2a092 None None] MFA Rules not processed for user `6e4225f72fbb4a06a638bbcd08aeacc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:27.787733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cbee626d-78e0-49e6-af27-48c0a3d2a092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:27.860985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1151/2286] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:27.882835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78065594-2ed2-4130-882a-ced7176cd86a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:27.884836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-78065594-2ed2-4130-882a-ced7176cd86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:27.962053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78065594-2ed2-4130-882a-ced7176cd86a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:27.966843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b97734-08eb-483b-b2fd-a3beaa9ab591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:27.967794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b97734-08eb-483b-b2fd-a3beaa9ab591 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:27.968167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b97734-08eb-483b-b2fd-a3beaa9ab591 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:27.968679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b97734-08eb-483b-b2fd-a3beaa9ab591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:27.980827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b97734-08eb-483b-b2fd-a3beaa9ab591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:27.982274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b97734-08eb-483b-b2fd-a3beaa9ab591 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:27.993937 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1136/2287] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:28.209132 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee626d-78e0-49e6-af27-48c0a3d2a092 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:28.215679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:28.219446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee626d-78e0-49e6-af27-48c0a3d2a092 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:28.223482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb17f56-ca95-40b6-bfc1-1562094859d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:28.224056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb17f56-ca95-40b6-bfc1-1562094859d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:28.226011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb17f56-ca95-40b6-bfc1-1562094859d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:28.226300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb17f56-ca95-40b6-bfc1-1562094859d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:28.294728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb17f56-ca95-40b6-bfc1-1562094859d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:28.296432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb17f56-ca95-40b6-bfc1-1562094859d9 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:28.307382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1152/2288] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:29.021837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b97734-08eb-483b-b2fd-a3beaa9ab591 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:29.076151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:29.097483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b97734-08eb-483b-b2fd-a3beaa9ab591 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:29.102870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:29.103799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:29.104093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:29.104358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] PATH_INFO: `/v3/users/c57384ab38a3408cbde3e0db0742ebcd` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:29.105545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57384ab38a3408cbde3e0db0742ebcd)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:29.108441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:29.412485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb17f56-ca95-40b6-bfc1-1562094859d9 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:29.414524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4bb17f56-ca95-40b6-bfc1-1562094859d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:29.433993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb17f56-ca95-40b6-bfc1-1562094859d9 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:29.438757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:29.439655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:29.439868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:29.440146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] PATH_INFO: `/v3/users/6be49bc260dc41f4993546f55174c18c` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:29.441369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6be49bc260dc41f4993546f55174c18c)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:29.443427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:29.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57384ab38a3408cbde3e0db0742ebcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:29.500905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57384ab38a3408cbde3e0db0742ebcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:29.585182 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57384ab38a3408cbde3e0db0742ebcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:29.604686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57384ab38a3408cbde3e0db0742ebcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:29.605975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-369c85ee-9bd7-4844-85d5-45dda5b2bce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57384ab38a3408cbde3e0db0742ebcd'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:29.622373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1137/2289] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:29 2022] DELETE /identity/v3/users/c57384ab38a3408cbde3e0db0742ebcd => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:29.627123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d508d3-b804-40cc-84e2-b99939a6fae0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:29.627688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d508d3-b804-40cc-84e2-b99939a6fae0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:29.628115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d508d3-b804-40cc-84e2-b99939a6fae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:29.654842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-64d508d3-b804-40cc-84e2-b99939a6fae0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:29.656183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-64d508d3-b804-40cc-84e2-b99939a6fae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:29.691051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6be49bc260dc41f4993546f55174c18c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:29.691849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6be49bc260dc41f4993546f55174c18c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:29.755965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1138/2290] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:29.766718 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d508d3-b804-40cc-84e2-b99939a6fae0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:29.769999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6be49bc260dc41f4993546f55174c18c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:29.775689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-64d508d3-b804-40cc-84e2-b99939a6fae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:29.786196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6be49bc260dc41f4993546f55174c18c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:29.787193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b2e48cc9-264a-41f9-bd7f-92e869793863 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6be49bc260dc41f4993546f55174c18c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:29.804684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1153/2291] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:29 2022] DELETE /identity/v3/users/6be49bc260dc41f4993546f55174c18c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:29.809733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb36165-e4b3-43a2-9390-0ad4be1788c4 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:29.810266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb36165-e4b3-43a2-9390-0ad4be1788c4 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:29.810619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb36165-e4b3-43a2-9390-0ad4be1788c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:29.850129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d508d3-b804-40cc-84e2-b99939a6fae0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:29.857232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-0eb36165-e4b3-43a2-9390-0ad4be1788c4 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:29.863243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0eb36165-e4b3-43a2-9390-0ad4be1788c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:29.872675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:29.928798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e6f0c-cf9c-40af-a8dd-cdba29ea8a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:29.933648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e6f0c-cf9c-40af-a8dd-cdba29ea8a51 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:29.933648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e6f0c-cf9c-40af-a8dd-cdba29ea8a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:29.933648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e6f0c-cf9c-40af-a8dd-cdba29ea8a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:29.942608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1154/2292] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:29.956962 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb36165-e4b3-43a2-9390-0ad4be1788c4 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:29.975714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb36165-e4b3-43a2-9390-0ad4be1788c4 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:29.980179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cc451-655c-4d6b-8f65-c74a0dc6c9d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:29.983724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cc451-655c-4d6b-8f65-c74a0dc6c9d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:29.983724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cc451-655c-4d6b-8f65-c74a0dc6c9d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:29.983724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cc451-655c-4d6b-8f65-c74a0dc6c9d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:29.996979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cc451-655c-4d6b-8f65-c74a0dc6c9d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:29.999846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cc451-655c-4d6b-8f65-c74a0dc6c9d2 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:30.012122 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e6f0c-cf9c-40af-a8dd-cdba29ea8a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:30.012800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1155/2293] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:30.013738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e6f0c-cf9c-40af-a8dd-cdba29ea8a51 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:30.026475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1139/2294] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:31.028741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cc451-655c-4d6b-8f65-c74a0dc6c9d2 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:31.030222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0c9cc451-655c-4d6b-8f65-c74a0dc6c9d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:31.038578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:31.044398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cc451-655c-4d6b-8f65-c74a0dc6c9d2 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:31.066239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e6f0c-cf9c-40af-a8dd-cdba29ea8a51 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:31.068848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-da1e6f0c-cf9c-40af-a8dd-cdba29ea8a51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:31.071772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:31.072944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:31.073375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:31.073815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] PATH_INFO: `/v3/projects/2f977c80e7514de8ae6bef0df5ee953b` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:31.075184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f977c80e7514de8ae6bef0df5ee953b)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:31.077380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:31.084764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e6f0c-cf9c-40af-a8dd-cdba29ea8a51 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:31.090151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:31.092060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:31.092060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:31.092060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] PATH_INFO: `/v3/projects/5defa47cffff44fdb446b0cb089d2029` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:31.093999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5defa47cffff44fdb446b0cb089d2029)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:31.096007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:31.322420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f977c80e7514de8ae6bef0df5ee953b', 'request_id': 'req-162d17a3-2cce-4fca-b923-76dffa2a01d7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:31.323642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5defa47cffff44fdb446b0cb089d2029', 'request_id': 'req-a31366ee-6d4f-4b45-b814-772e98ce5099'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:31.329120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f977c80e7514de8ae6bef0df5ee953b', 'request_id': 'req-162d17a3-2cce-4fca-b923-76dffa2a01d7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:31.331074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5defa47cffff44fdb446b0cb089d2029', 'request_id': 'req-a31366ee-6d4f-4b45-b814-772e98ce5099'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:31.406988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5defa47cffff44fdb446b0cb089d2029', 'request_id': 'req-a31366ee-6d4f-4b45-b814-772e98ce5099'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:31.429974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] The token cache is being invalidate because project 5defa47cffff44fdb446b0cb089d2029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:31.430284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:31.431013 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a31366ee-6d4f-4b45-b814-772e98ce5099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:31.432566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1140/2295] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:31 2022] DELETE /identity/v3/projects/5defa47cffff44fdb446b0cb089d2029 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:31.433979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c373-fe02-4ff4-8cff-b8f5807b6438 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:31.434260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c373-fe02-4ff4-8cff-b8f5807b6438 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:31.434528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c373-fe02-4ff4-8cff-b8f5807b6438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:31.456077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-85a0c373-fe02-4ff4-8cff-b8f5807b6438 None None] MFA Rules not processed for user `de2273220e824339bd13d968c75df26b`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:31.457536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-85a0c373-fe02-4ff4-8cff-b8f5807b6438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:31.517016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:31.523378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f977c80e7514de8ae6bef0df5ee953b', 'request_id': 'req-162d17a3-2cce-4fca-b923-76dffa2a01d7'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:31.533569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] The token cache is being invalidate because project 2f977c80e7514de8ae6bef0df5ee953b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:31.533865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:31.534637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:31.540369 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1156/2296] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:31 2022] DELETE /identity/v3/projects/2f977c80e7514de8ae6bef0df5ee953b => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:31.541815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:31.543606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:31.547998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1141/2297] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:31.549361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4145f-fd56-40c7-8cba-b30c4cb3e849 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:31.549639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4145f-fd56-40c7-8cba-b30c4cb3e849 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:31.549937 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4145f-fd56-40c7-8cba-b30c4cb3e849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:31.574336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7ca4145f-fd56-40c7-8cba-b30c4cb3e849 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:31.587400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7ca4145f-fd56-40c7-8cba-b30c4cb3e849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:31.626709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d17a3-2cce-4fca-b923-76dffa2a01d7 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:31.734286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:31.734952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:31.735457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:31.735814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] PATH_INFO: `/v3/users/6e4225f72fbb4a06a638bbcd08aeacc9` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:31.737034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4225f72fbb4a06a638bbcd08aeacc9)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:31.739784 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1142/2298] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:31.743288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:31.748971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437a9c-6996-45eb-a553-f6cb59a10b3b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:31.748971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437a9c-6996-45eb-a553-f6cb59a10b3b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:31.748971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437a9c-6996-45eb-a553-f6cb59a10b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:31.773005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ab437a9c-6996-45eb-a553-f6cb59a10b3b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:31.774230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ab437a9c-6996-45eb-a553-f6cb59a10b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:31.783430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1143/2299] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:31.788364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab437a9c-6996-45eb-a553-f6cb59a10b3b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:31.799567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab437a9c-6996-45eb-a553-f6cb59a10b3b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:31.806296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cff99a-c1cc-4e0a-a18c-895c26717e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:31.807224 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cff99a-c1cc-4e0a-a18c-895c26717e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:31.807498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cff99a-c1cc-4e0a-a18c-895c26717e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:31.807792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cff99a-c1cc-4e0a-a18c-895c26717e74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:31.808455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cff99a-c1cc-4e0a-a18c-895c26717e74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:31.808808 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cff99a-c1cc-4e0a-a18c-895c26717e74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:31.811062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cff99a-c1cc-4e0a-a18c-895c26717e74 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:31.829569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1144/2300] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:31:31.834694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cff99a-c1cc-4e0a-a18c-895c26717e74 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:31.848062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cff99a-c1cc-4e0a-a18c-895c26717e74 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:31.865364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d5445-1d1d-43e9-bf0b-1c8958fb6edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:31.865364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d5445-1d1d-43e9-bf0b-1c8958fb6edf admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:31.865364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d5445-1d1d-43e9-bf0b-1c8958fb6edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:31.865364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d5445-1d1d-43e9-bf0b-1c8958fb6edf admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:31.866741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d5445-1d1d-43e9-bf0b-1c8958fb6edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:31.866741 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d5445-1d1d-43e9-bf0b-1c8958fb6edf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:31.974648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1145/2301] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:31 2022] POST /identity/v3/projects => generated 389 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:31.979866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d5445-1d1d-43e9-bf0b-1c8958fb6edf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:32.004151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4225f72fbb4a06a638bbcd08aeacc9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:32.007803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4225f72fbb4a06a638bbcd08aeacc9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:32.083489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d5445-1d1d-43e9-bf0b-1c8958fb6edf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:32.089902 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a7190-3f80-464c-ac0d-ba28389e13ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:32.090649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a7190-3f80-464c-ac0d-ba28389e13ab admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:32.091163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a7190-3f80-464c-ac0d-ba28389e13ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:32.091521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a7190-3f80-464c-ac0d-ba28389e13ab admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:32.092262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a7190-3f80-464c-ac0d-ba28389e13ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:32.094260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a7190-3f80-464c-ac0d-ba28389e13ab admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:32.133279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4225f72fbb4a06a638bbcd08aeacc9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:32.169937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4225f72fbb4a06a638bbcd08aeacc9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:32.172899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4225f72fbb4a06a638bbcd08aeacc9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:32.191130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1157/2302] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:31 2022] DELETE /identity/v3/users/6e4225f72fbb4a06a638bbcd08aeacc9 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:32.266669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1146/2303] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:31 2022] POST /identity/v3/users => generated 295 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:32.270411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:32.272452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:32.342196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55c9e9-a986-4fb1-9793-36b025c5de69 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:32.346885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70584c27-c217-48ad-a773-bfda978d9350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:32.347509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70584c27-c217-48ad-a773-bfda978d9350 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:32.347747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70584c27-c217-48ad-a773-bfda978d9350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:32.348018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70584c27-c217-48ad-a773-bfda978d9350 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:32.348685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70584c27-c217-48ad-a773-bfda978d9350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:32.350844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70584c27-c217-48ad-a773-bfda978d9350 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:32.362017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1158/2304] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:32.371128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a7190-3f80-464c-ac0d-ba28389e13ab admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:32.376314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a7190-3f80-464c-ac0d-ba28389e13ab admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:32.381162 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b7cce-5d2d-461c-a26d-c140bd8191c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:32.381802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b7cce-5d2d-461c-a26d-c140bd8191c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:32.382074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b7cce-5d2d-461c-a26d-c140bd8191c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:32.382342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b7cce-5d2d-461c-a26d-c140bd8191c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:32.383005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b7cce-5d2d-461c-a26d-c140bd8191c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:32.384752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b7cce-5d2d-461c-a26d-c140bd8191c1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:32.395275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:32.397333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1147/2305] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:32.400326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70584c27-c217-48ad-a773-bfda978d9350 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:32.410673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70584c27-c217-48ad-a773-bfda978d9350 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:32.416115 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e0ca9-f959-4986-95b2-b41b86cf24c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:32.416782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e0ca9-f959-4986-95b2-b41b86cf24c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:32.417066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e0ca9-f959-4986-95b2-b41b86cf24c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:32.417345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e0ca9-f959-4986-95b2-b41b86cf24c3 admin admin] PATH_INFO: `/v3/projects/9fe7d55243ed4fb395edbda2ee620f35/users/4993091d3f504f88b25769cc34ab5623/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:32.432972 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e0ca9-f959-4986-95b2-b41b86cf24c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe7d55243ed4fb395edbda2ee620f35, user_id=4993091d3f504f88b25769cc34ab5623, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:32.435360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e0ca9-f959-4986-95b2-b41b86cf24c3 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:32.555793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1159/2306] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:32 2022] PUT /identity/v3/projects/9fe7d55243ed4fb395edbda2ee620f35/users/4993091d3f504f88b25769cc34ab5623/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:32.563997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992072b-ab23-41c9-94f1-9cc587b95da6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:32.564262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992072b-ab23-41c9-94f1-9cc587b95da6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:32.564541 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992072b-ab23-41c9-94f1-9cc587b95da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:32.597891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4992072b-ab23-41c9-94f1-9cc587b95da6 None None] MFA Rules not processed for user `4993091d3f504f88b25769cc34ab5623`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:32.599295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4992072b-ab23-41c9-94f1-9cc587b95da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:32.685593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1148/2307] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:32.699301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e0ca9-f959-4986-95b2-b41b86cf24c3 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:32.701096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d83e0ca9-f959-4986-95b2-b41b86cf24c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:32.762960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e0ca9-f959-4986-95b2-b41b86cf24c3 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:32.766875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466781-2bc3-4954-a671-29488f0d09dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:32.767732 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466781-2bc3-4954-a671-29488f0d09dd service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:32.768009 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466781-2bc3-4954-a671-29488f0d09dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:32.768279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466781-2bc3-4954-a671-29488f0d09dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:32.779379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992072b-ab23-41c9-94f1-9cc587b95da6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:32.782959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4992072b-ab23-41c9-94f1-9cc587b95da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:32.792627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2466781-2bc3-4954-a671-29488f0d09dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:32.794156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2466781-2bc3-4954-a671-29488f0d09dd service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:32.809097 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1160/2308] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:32.886872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992072b-ab23-41c9-94f1-9cc587b95da6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:32.894705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:32.894705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:32.894705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:32.895403 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] PATH_INFO: `/v3/projects/5f899b7bf2b6427797fb2b4ced4c0340` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:32.897264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f899b7bf2b6427797fb2b4ced4c0340)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:32.900042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:32.973027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:33.076326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f899b7bf2b6427797fb2b4ced4c0340', 'request_id': 'req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:33.084221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f899b7bf2b6427797fb2b4ced4c0340', 'request_id': 'req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:33.154916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466781-2bc3-4954-a671-29488f0d09dd service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:33.156502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b2466781-2bc3-4954-a671-29488f0d09dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:33.171764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466781-2bc3-4954-a671-29488f0d09dd service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:33.229061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f899b7bf2b6427797fb2b4ced4c0340', 'request_id': 'req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:33.248504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] The token cache is being invalidate because project 5f899b7bf2b6427797fb2b4ced4c0340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:33.248797 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:33.249444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-625f5ede-b9ac-4b52-95a0-e8b9a3c48000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:33.251383 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1149/2309] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:32 2022] DELETE /identity/v3/projects/5f899b7bf2b6427797fb2b4ced4c0340 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:33.255661 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:33.261136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df742b8-9e61-49b9-9759-6bfc5909107e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.261277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.261389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df742b8-9e61-49b9-9759-6bfc5909107e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.261480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.261480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] PATH_INFO: `/v3/users/381213869d604763aa4b36d5b34960d8` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.261648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df742b8-9e61-49b9-9759-6bfc5909107e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.261722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=381213869d604763aa4b36d5b34960d8)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:33.261815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:33.281067 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5df742b8-9e61-49b9-9759-6bfc5909107e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:33.282299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5df742b8-9e61-49b9-9759-6bfc5909107e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:33.293992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1150/2310] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:33.303008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52282ee-639b-4598-a0d6-b71ad773fc8f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.303292 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52282ee-639b-4598-a0d6-b71ad773fc8f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.303599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52282ee-639b-4598-a0d6-b71ad773fc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.324686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e52282ee-639b-4598-a0d6-b71ad773fc8f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:33.326022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e52282ee-639b-4598-a0d6-b71ad773fc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:33.336580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1151/2311] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:33.343763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52282ee-639b-4598-a0d6-b71ad773fc8f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:33.355905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52282ee-639b-4598-a0d6-b71ad773fc8f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:33.361475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2ddc1-9273-4b9a-9b52-7e8938c256a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:33.362423 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2ddc1-9273-4b9a-9b52-7e8938c256a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.362803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2ddc1-9273-4b9a-9b52-7e8938c256a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.363239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2ddc1-9273-4b9a-9b52-7e8938c256a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.364082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2ddc1-9273-4b9a-9b52-7e8938c256a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:33.364453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2ddc1-9273-4b9a-9b52-7e8938c256a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:33.366244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2ddc1-9273-4b9a-9b52-7e8938c256a9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:33.382270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1152/2312] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:31:33.392090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2ddc1-9273-4b9a-9b52-7e8938c256a9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:33.403791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2ddc1-9273-4b9a-9b52-7e8938c256a9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:33.408730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a58e8d-5c01-422d-8fdc-920ced92d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:33.409352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a58e8d-5c01-422d-8fdc-920ced92d71e admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.409637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a58e8d-5c01-422d-8fdc-920ced92d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.409911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a58e8d-5c01-422d-8fdc-920ced92d71e admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.410582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a58e8d-5c01-422d-8fdc-920ced92d71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:33.412756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a58e8d-5c01-422d-8fdc-920ced92d71e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:33.549541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381213869d604763aa4b36d5b34960d8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:33.550729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381213869d604763aa4b36d5b34960d8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:33.599888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1153/2313] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:33 2022] POST /identity/v3/projects => generated 371 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:33.604560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a58e8d-5c01-422d-8fdc-920ced92d71e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:33.606338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e1a58e8d-5c01-422d-8fdc-920ced92d71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:33.689020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a58e8d-5c01-422d-8fdc-920ced92d71e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:33.695613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6f841-0bdc-46e0-8531-9a3287aae391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:33.696252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6f841-0bdc-46e0-8531-9a3287aae391 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.696533 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6f841-0bdc-46e0-8531-9a3287aae391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.696828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6f841-0bdc-46e0-8531-9a3287aae391 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.697475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6f841-0bdc-46e0-8531-9a3287aae391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:33.699463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6f841-0bdc-46e0-8531-9a3287aae391 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:33.709188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381213869d604763aa4b36d5b34960d8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:33.722440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381213869d604763aa4b36d5b34960d8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:33.723223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1312bd5d-7715-49c6-a302-c11ea4b4c9c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381213869d604763aa4b36d5b34960d8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:33.742027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1161/2314] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:33 2022] DELETE /identity/v3/users/381213869d604763aa4b36d5b34960d8 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:33.746774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f1d84-1e54-4aab-930c-16ea1f0c4875 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.747095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f1d84-1e54-4aab-930c-16ea1f0c4875 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.747421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f1d84-1e54-4aab-930c-16ea1f0c4875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.779717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d00f1d84-1e54-4aab-930c-16ea1f0c4875 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:33.781429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d00f1d84-1e54-4aab-930c-16ea1f0c4875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:33.836159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1154/2315] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:33 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:33.844985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1162/2316] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:33.845729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6f841-0bdc-46e0-8531-9a3287aae391 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:33.855395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f1d84-1e54-4aab-930c-16ea1f0c4875 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:33.856737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6f841-0bdc-46e0-8531-9a3287aae391 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:33.857160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d00f1d84-1e54-4aab-930c-16ea1f0c4875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:33.861489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c13116-d05a-4133-9077-eb9823ca70a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:33.862103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13116-d05a-4133-9077-eb9823ca70a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.862412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13116-d05a-4133-9077-eb9823ca70a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.862857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13116-d05a-4133-9077-eb9823ca70a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.863514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c13116-d05a-4133-9077-eb9823ca70a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:33.865747 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c13116-d05a-4133-9077-eb9823ca70a0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:33.877424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1155/2317] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:33.882058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c13116-d05a-4133-9077-eb9823ca70a0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:33.897735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c13116-d05a-4133-9077-eb9823ca70a0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:33.903618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03582df-b311-49fa-9a9f-244d54c83bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:33.904396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03582df-b311-49fa-9a9f-244d54c83bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.904796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03582df-b311-49fa-9a9f-244d54c83bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.905293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03582df-b311-49fa-9a9f-244d54c83bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.906039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03582df-b311-49fa-9a9f-244d54c83bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:33.907869 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03582df-b311-49fa-9a9f-244d54c83bbd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:33.920335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1156/2318] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:33.924004 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03582df-b311-49fa-9a9f-244d54c83bbd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:33.931802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f1d84-1e54-4aab-930c-16ea1f0c4875 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:33.935755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03582df-b311-49fa-9a9f-244d54c83bbd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:33.937694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757f326-b8b1-4278-9584-6a6efa91d0f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:33.938485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757f326-b8b1-4278-9584-6a6efa91d0f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.938793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757f326-b8b1-4278-9584-6a6efa91d0f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.941229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757f326-b8b1-4278-9584-6a6efa91d0f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.941424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7feed-fbd3-4f72-8c78-4afa034628c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:33.941992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7feed-fbd3-4f72-8c78-4afa034628c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:33.942275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7feed-fbd3-4f72-8c78-4afa034628c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:33.942550 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7feed-fbd3-4f72-8c78-4afa034628c7 admin admin] PATH_INFO: `/v3/projects/9cd6ab1bf94c41879b0302dd3da8e08e/users/f3159df2e3774992962cd1f7cb26df58/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:33.953177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757f326-b8b1-4278-9584-6a6efa91d0f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:33.954996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757f326-b8b1-4278-9584-6a6efa91d0f3 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:33.959748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7feed-fbd3-4f72-8c78-4afa034628c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd6ab1bf94c41879b0302dd3da8e08e, user_id=f3159df2e3774992962cd1f7cb26df58, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:33.962424 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7feed-fbd3-4f72-8c78-4afa034628c7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:33.965326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1163/2319] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:34.010240 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:34.125186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:34.133218 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1157/2320] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:33 2022] PUT /identity/v3/projects/9cd6ab1bf94c41879b0302dd3da8e08e/users/f3159df2e3774992962cd1f7cb26df58/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:34.144014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e9a57-7484-4a61-8223-91f157ce0c21 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:34.144516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e9a57-7484-4a61-8223-91f157ce0c21 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:34.146153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e9a57-7484-4a61-8223-91f157ce0c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:34.179153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-017e9a57-7484-4a61-8223-91f157ce0c21 None None] MFA Rules not processed for user `f3159df2e3774992962cd1f7cb26df58`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:34.184459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-017e9a57-7484-4a61-8223-91f157ce0c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:34.271913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1164/2321] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:34.283320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7feed-fbd3-4f72-8c78-4afa034628c7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:34.285690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-49e7feed-fbd3-4f72-8c78-4afa034628c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:34.373713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7feed-fbd3-4f72-8c78-4afa034628c7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:34.377857 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29283f8-b4e5-47f7-b0e5-bd10d6cc74c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:34.378575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29283f8-b4e5-47f7-b0e5-bd10d6cc74c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:34.378848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29283f8-b4e5-47f7-b0e5-bd10d6cc74c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:34.379117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29283f8-b4e5-47f7-b0e5-bd10d6cc74c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:34.390542 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29283f8-b4e5-47f7-b0e5-bd10d6cc74c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:34.392063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29283f8-b4e5-47f7-b0e5-bd10d6cc74c4 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:34.404742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1158/2322] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:34.429237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e9a57-7484-4a61-8223-91f157ce0c21 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:34.495411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e9a57-7484-4a61-8223-91f157ce0c21 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:34.500101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:34.500748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:34.501104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:34.501315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] PATH_INFO: `/v3/users/f3159df2e3774992962cd1f7cb26df58` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:34.502495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3159df2e3774992962cd1f7cb26df58)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:34.504486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:34.526574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29283f8-b4e5-47f7-b0e5-bd10d6cc74c4 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:34.529724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b29283f8-b4e5-47f7-b0e5-bd10d6cc74c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:34.551461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29283f8-b4e5-47f7-b0e5-bd10d6cc74c4 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:34.559622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:34.560630 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:34.560941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:34.561167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] PATH_INFO: `/v3/projects/a37d5b89b81045679105900f317baabe` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:34.562378 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37d5b89b81045679105900f317baabe)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:34.564626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:34.749602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37d5b89b81045679105900f317baabe', 'request_id': 'req-2386156f-eb9e-4ca8-a810-722210da2c0f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:34.751955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3159df2e3774992962cd1f7cb26df58'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:34.752708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3159df2e3774992962cd1f7cb26df58'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:34.789765 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37d5b89b81045679105900f317baabe', 'request_id': 'req-2386156f-eb9e-4ca8-a810-722210da2c0f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:34.838755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3159df2e3774992962cd1f7cb26df58'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:34.853831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3159df2e3774992962cd1f7cb26df58'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:34.854972 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab668a05-9b88-481d-9945-07d607734d80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3159df2e3774992962cd1f7cb26df58'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:34.869118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1165/2323] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:34 2022] DELETE /identity/v3/users/f3159df2e3774992962cd1f7cb26df58 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:34.873615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa9f1f-69ab-4706-adca-df5093b35d4f None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:34.873899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa9f1f-69ab-4706-adca-df5093b35d4f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:34.874176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa9f1f-69ab-4706-adca-df5093b35d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:34.895845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-53fa9f1f-69ab-4706-adca-df5093b35d4f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:34.897048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-53fa9f1f-69ab-4706-adca-df5093b35d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:34.927507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37d5b89b81045679105900f317baabe', 'request_id': 'req-2386156f-eb9e-4ca8-a810-722210da2c0f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:34.943612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] The token cache is being invalidate because project a37d5b89b81045679105900f317baabe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:34.944317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:34.945643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2386156f-eb9e-4ca8-a810-722210da2c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:34.948061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1159/2324] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:34 2022] DELETE /identity/v3/projects/a37d5b89b81045679105900f317baabe => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:34.958549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f31f0-e14c-487f-bd4e-bd5b2832f526 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:34.959623 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f31f0-e14c-487f-bd4e-bd5b2832f526 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:34.960261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f31f0-e14c-487f-bd4e-bd5b2832f526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:34.976842 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1166/2325] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:34.982980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9a3f31f0-e14c-487f-bd4e-bd5b2832f526 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:34.984234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9a3f31f0-e14c-487f-bd4e-bd5b2832f526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:34.986893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa9f1f-69ab-4706-adca-df5093b35d4f None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:34.988642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-53fa9f1f-69ab-4706-adca-df5093b35d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:34.994244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1160/2326] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:35.003444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8850a93-2a4f-413b-bcdb-b7a8e9283f3f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.003756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8850a93-2a4f-413b-bcdb-b7a8e9283f3f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.004030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8850a93-2a4f-413b-bcdb-b7a8e9283f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.026724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b8850a93-2a4f-413b-bcdb-b7a8e9283f3f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:35.028040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b8850a93-2a4f-413b-bcdb-b7a8e9283f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:35.042133 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1161/2327] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:35.046381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8850a93-2a4f-413b-bcdb-b7a8e9283f3f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:35.056209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8850a93-2a4f-413b-bcdb-b7a8e9283f3f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:35.058321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa9f1f-69ab-4706-adca-df5093b35d4f None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:35.061345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb544078-5413-4ebd-8d7d-5df70d507322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:35.061968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544078-5413-4ebd-8d7d-5df70d507322 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.062239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544078-5413-4ebd-8d7d-5df70d507322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.062512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544078-5413-4ebd-8d7d-5df70d507322 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.063151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0947b-fe67-43f0-98f8-d00da4d07662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:35.063279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb544078-5413-4ebd-8d7d-5df70d507322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:35.063510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb544078-5413-4ebd-8d7d-5df70d507322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:35.070066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0947b-fe67-43f0-98f8-d00da4d07662 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.070066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0947b-fe67-43f0-98f8-d00da4d07662 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.070362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb544078-5413-4ebd-8d7d-5df70d507322 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:35.070457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0947b-fe67-43f0-98f8-d00da4d07662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.080978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1162/2328] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:31:35.084993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0947b-fe67-43f0-98f8-d00da4d07662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:35.086617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0947b-fe67-43f0-98f8-d00da4d07662 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:35.087564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb544078-5413-4ebd-8d7d-5df70d507322 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:35.102492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb544078-5413-4ebd-8d7d-5df70d507322 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:35.103124 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1167/2329] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:35.108086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c1ee2-c834-42c2-a64c-4a75b6b62e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:35.108699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c1ee2-c834-42c2-a64c-4a75b6b62e55 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.108975 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c1ee2-c834-42c2-a64c-4a75b6b62e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.109244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c1ee2-c834-42c2-a64c-4a75b6b62e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.109879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c1ee2-c834-42c2-a64c-4a75b6b62e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:35.112253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c1ee2-c834-42c2-a64c-4a75b6b62e55 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:35.223821 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c4b79-0156-4f7a-b9bb-961028a44629 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.224582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c4b79-0156-4f7a-b9bb-961028a44629 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.224582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c4b79-0156-4f7a-b9bb-961028a44629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.250414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-6c7c4b79-0156-4f7a-b9bb-961028a44629 None None] MFA Rules not processed for user `e7a29212dfae4a93beea7d11085cde63`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:35.252101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6c7c4b79-0156-4f7a-b9bb-961028a44629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:35.259335 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:35.267767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1163/2330] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:35 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:35.268298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c1ee2-c834-42c2-a64c-4a75b6b62e55 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:35.325420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1168/2331] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:35.343934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c1ee2-c834-42c2-a64c-4a75b6b62e55 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:35.352124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540c1c0-cdca-4484-8cdf-1bf66a8d72a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:35.353000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540c1c0-cdca-4484-8cdf-1bf66a8d72a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.353565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540c1c0-cdca-4484-8cdf-1bf66a8d72a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.353934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540c1c0-cdca-4484-8cdf-1bf66a8d72a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.354583 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540c1c0-cdca-4484-8cdf-1bf66a8d72a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:35.356953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540c1c0-cdca-4484-8cdf-1bf66a8d72a8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:35.368784 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:35.499774 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1164/2332] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:35 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:35.507379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c4b79-0156-4f7a-b9bb-961028a44629 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:35.538684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c4b79-0156-4f7a-b9bb-961028a44629 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:35.554122 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb03019-c0ac-4aa3-865e-c1c7061d50d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:35.555168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03019-c0ac-4aa3-865e-c1c7061d50d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.555654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03019-c0ac-4aa3-865e-c1c7061d50d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.556353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03019-c0ac-4aa3-865e-c1c7061d50d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.557231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb03019-c0ac-4aa3-865e-c1c7061d50d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:35.560751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb03019-c0ac-4aa3-865e-c1c7061d50d1 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:35.595380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1169/2333] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:35.605665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540c1c0-cdca-4484-8cdf-1bf66a8d72a8 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:35.616994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540c1c0-cdca-4484-8cdf-1bf66a8d72a8 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:35.622082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73647ef7-0fc7-4dd0-9aa4-1a9e88e20c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:35.622706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73647ef7-0fc7-4dd0-9aa4-1a9e88e20c48 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.622987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73647ef7-0fc7-4dd0-9aa4-1a9e88e20c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.623258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73647ef7-0fc7-4dd0-9aa4-1a9e88e20c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.624542 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73647ef7-0fc7-4dd0-9aa4-1a9e88e20c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:35.626155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73647ef7-0fc7-4dd0-9aa4-1a9e88e20c48 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:35.641679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1165/2334] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:35.647975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb03019-c0ac-4aa3-865e-c1c7061d50d1 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:35.666961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb03019-c0ac-4aa3-865e-c1c7061d50d1 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:35.678846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0f1d8-a81b-4bb9-aea8-bff6fd112c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:35.680568 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0f1d8-a81b-4bb9-aea8-bff6fd112c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.684714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0f1d8-a81b-4bb9-aea8-bff6fd112c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.684714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0f1d8-a81b-4bb9-aea8-bff6fd112c4b admin admin] PATH_INFO: `/v3/projects/60613c7205f34175980c3823e6cd5baa/users/b3f78ffea4704c539d948e831f6a0b62/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.700839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0f1d8-a81b-4bb9-aea8-bff6fd112c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60613c7205f34175980c3823e6cd5baa, user_id=b3f78ffea4704c539d948e831f6a0b62, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:35.709477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0f1d8-a81b-4bb9-aea8-bff6fd112c4b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:35.730297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73647ef7-0fc7-4dd0-9aa4-1a9e88e20c48 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:35.732321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-73647ef7-0fc7-4dd0-9aa4-1a9e88e20c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:35.756995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73647ef7-0fc7-4dd0-9aa4-1a9e88e20c48 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:35.762550 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:35.763147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.763425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.763957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] PATH_INFO: `/v3/projects/9cd6ab1bf94c41879b0302dd3da8e08e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.765310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd6ab1bf94c41879b0302dd3da8e08e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:35.767580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:35.797540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1170/2335] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:35 2022] PUT /identity/v3/projects/60613c7205f34175980c3823e6cd5baa/users/b3f78ffea4704c539d948e831f6a0b62/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:35.810613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdff38-1d23-44e6-94be-c8333c3e3eba None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:35.811168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdff38-1d23-44e6-94be-c8333c3e3eba None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:35.811876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdff38-1d23-44e6-94be-c8333c3e3eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:35.848357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-f7fdff38-1d23-44e6-94be-c8333c3e3eba None None] MFA Rules not processed for user `b3f78ffea4704c539d948e831f6a0b62`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:35.850890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f7fdff38-1d23-44e6-94be-c8333c3e3eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:35.922134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd6ab1bf94c41879b0302dd3da8e08e', 'request_id': 'req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:35.931330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd6ab1bf94c41879b0302dd3da8e08e', 'request_id': 'req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:35.947761 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1171/2336] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:35.952603 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdff38-1d23-44e6-94be-c8333c3e3eba None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.034870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd6ab1bf94c41879b0302dd3da8e08e', 'request_id': 'req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:36.041227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdff38-1d23-44e6-94be-c8333c3e3eba None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.054985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] The token cache is being invalidate because project 9cd6ab1bf94c41879b0302dd3da8e08e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:31:36.055379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:36.056140 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-fed04dda-dbb7-4c7a-b1ae-930bef9c81b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:36.061397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1166/2337] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:31:35 2022] DELETE /identity/v3/projects/9cd6ab1bf94c41879b0302dd3da8e08e => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:36.063289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0582cc-38ff-42a6-8329-1d0d9a78ffe1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.070796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0582cc-38ff-42a6-8329-1d0d9a78ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.071502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0582cc-38ff-42a6-8329-1d0d9a78ffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.117448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ee0582cc-38ff-42a6-8329-1d0d9a78ffe1 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:36.118864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ee0582cc-38ff-42a6-8329-1d0d9a78ffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:36.142478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c2e3d-e05b-4533-8834-70d3a818ea00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.142854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c2e3d-e05b-4533-8834-70d3a818ea00 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.143186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c2e3d-e05b-4533-8834-70d3a818ea00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.143463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c2e3d-e05b-4533-8834-70d3a818ea00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.144141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c2e3d-e05b-4533-8834-70d3a818ea00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.147592 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c2e3d-e05b-4533-8834-70d3a818ea00 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.191223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1167/2338] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:36.200912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d432dd-8536-45de-8bd0-ad065b73f3c1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.201543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d432dd-8536-45de-8bd0-ad065b73f3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.202038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d432dd-8536-45de-8bd0-ad065b73f3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.228929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c9d432dd-8536-45de-8bd0-ad065b73f3c1 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:36.231332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c9d432dd-8536-45de-8bd0-ad065b73f3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:36.242132 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1168/2339] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:36.248436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d432dd-8536-45de-8bd0-ad065b73f3c1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.252675 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d432dd-8536-45de-8bd0-ad065b73f3c1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.257940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74937163-83fd-4dc2-8ebb-fbb77bb835bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.258545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74937163-83fd-4dc2-8ebb-fbb77bb835bb admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.258824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74937163-83fd-4dc2-8ebb-fbb77bb835bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.259096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74937163-83fd-4dc2-8ebb-fbb77bb835bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.259798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74937163-83fd-4dc2-8ebb-fbb77bb835bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:31:36.260086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74937163-83fd-4dc2-8ebb-fbb77bb835bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.261700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74937163-83fd-4dc2-8ebb-fbb77bb835bb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.275790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1169/2340] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:31:36.280840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74937163-83fd-4dc2-8ebb-fbb77bb835bb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.281278 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1172/2341] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:35 2022] POST /identity/v3/projects => generated 385 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:36.285204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c2e3d-e05b-4533-8834-70d3a818ea00 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.286905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d73c2e3d-e05b-4533-8834-70d3a818ea00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:36.291280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74937163-83fd-4dc2-8ebb-fbb77bb835bb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.351929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c8027-844a-4fd6-9b6f-ac9d57466e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.352540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c8027-844a-4fd6-9b6f-ac9d57466e04 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.352823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c8027-844a-4fd6-9b6f-ac9d57466e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.353094 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c8027-844a-4fd6-9b6f-ac9d57466e04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.353983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c8027-844a-4fd6-9b6f-ac9d57466e04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.355916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c8027-844a-4fd6-9b6f-ac9d57466e04 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.356043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c2e3d-e05b-4533-8834-70d3a818ea00 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.360758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457d5f7-e968-411f-b65d-7c5b6252b998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.361375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457d5f7-e968-411f-b65d-7c5b6252b998 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.361650 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457d5f7-e968-411f-b65d-7c5b6252b998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.361917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457d5f7-e968-411f-b65d-7c5b6252b998 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.362541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457d5f7-e968-411f-b65d-7c5b6252b998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.364573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457d5f7-e968-411f-b65d-7c5b6252b998 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.478176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1173/2342] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:36 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:36.478641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1170/2343] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:36 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:36.479950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457d5f7-e968-411f-b65d-7c5b6252b998 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.480394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c8027-844a-4fd6-9b6f-ac9d57466e04 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.489905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457d5f7-e968-411f-b65d-7c5b6252b998 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.547059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c8027-844a-4fd6-9b6f-ac9d57466e04 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.551751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be9230-1a54-4f4d-866c-de4bc4aff858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.552084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c830a15-11a1-4882-a04b-fd6bd5df374c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.552375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be9230-1a54-4f4d-866c-de4bc4aff858 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.553379 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c830a15-11a1-4882-a04b-fd6bd5df374c admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.553562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be9230-1a54-4f4d-866c-de4bc4aff858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.553703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c830a15-11a1-4882-a04b-fd6bd5df374c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.553878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be9230-1a54-4f4d-866c-de4bc4aff858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.554038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c830a15-11a1-4882-a04b-fd6bd5df374c admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.554352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be9230-1a54-4f4d-866c-de4bc4aff858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.554613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c830a15-11a1-4882-a04b-fd6bd5df374c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.555961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be9230-1a54-4f4d-866c-de4bc4aff858 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.556481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c830a15-11a1-4882-a04b-fd6bd5df374c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.570099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1174/2344] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:36.571719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be9230-1a54-4f4d-866c-de4bc4aff858 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.581586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be9230-1a54-4f4d-866c-de4bc4aff858 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.586689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c644d5-162e-4899-b745-f11582d1cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.588306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c644d5-162e-4899-b745-f11582d1cfce admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.588306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c644d5-162e-4899-b745-f11582d1cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.588306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c644d5-162e-4899-b745-f11582d1cfce admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.589152 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c644d5-162e-4899-b745-f11582d1cfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.590125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c644d5-162e-4899-b745-f11582d1cfce admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.602294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1175/2345] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:36.606696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c644d5-162e-4899-b745-f11582d1cfce admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.618047 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c644d5-162e-4899-b745-f11582d1cfce admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.622984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb1489-b4c4-4fc4-8407-732f98690b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.623687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb1489-b4c4-4fc4-8407-732f98690b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.623968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb1489-b4c4-4fc4-8407-732f98690b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.624248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb1489-b4c4-4fc4-8407-732f98690b8f admin admin] PATH_INFO: `/v3/projects/2fc699337d0b407aa2aceebcf116671e/users/e18d3231ba03446cac59dcbd1a512628/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.643201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb1489-b4c4-4fc4-8407-732f98690b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fc699337d0b407aa2aceebcf116671e, user_id=e18d3231ba03446cac59dcbd1a512628, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.645976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb1489-b4c4-4fc4-8407-732f98690b8f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.703428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1171/2346] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:36 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:36.707177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c830a15-11a1-4882-a04b-fd6bd5df374c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.724751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c830a15-11a1-4882-a04b-fd6bd5df374c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.730824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabe4a3-bc10-4a41-a2c8-84f242a00d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.731597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabe4a3-bc10-4a41-a2c8-84f242a00d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.732265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabe4a3-bc10-4a41-a2c8-84f242a00d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.732543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabe4a3-bc10-4a41-a2c8-84f242a00d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.733199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabe4a3-bc10-4a41-a2c8-84f242a00d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.738338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1176/2347] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:36 2022] PUT /identity/v3/projects/2fc699337d0b407aa2aceebcf116671e/users/e18d3231ba03446cac59dcbd1a512628/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:36.750289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabe4a3-bc10-4a41-a2c8-84f242a00d50 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.765392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf6efd-3682-444a-b16e-10399d149ca1 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.766034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf6efd-3682-444a-b16e-10399d149ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.767215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf6efd-3682-444a-b16e-10399d149ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.783826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1172/2348] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:36.784750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabe4a3-bc10-4a41-a2c8-84f242a00d50 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.796903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabe4a3-bc10-4a41-a2c8-84f242a00d50 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.827573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ffbf6efd-3682-444a-b16e-10399d149ca1 None None] MFA Rules not processed for user `e18d3231ba03446cac59dcbd1a512628`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:36.827573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ffbf6efd-3682-444a-b16e-10399d149ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:36.925192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:36.928689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fddd15-424e-4b1d-85ff-8f9c25a51d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.929719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fddd15-424e-4b1d-85ff-8f9c25a51d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.930159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fddd15-424e-4b1d-85ff-8f9c25a51d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.930586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fddd15-424e-4b1d-85ff-8f9c25a51d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.932153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fddd15-424e-4b1d-85ff-8f9c25a51d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.935232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fddd15-424e-4b1d-85ff-8f9c25a51d03 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.941764 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1177/2349] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 05 12:31:36.946584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf6efd-3682-444a-b16e-10399d149ca1 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.953293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:36.953293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1173/2350] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:36.962494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fddd15-424e-4b1d-85ff-8f9c25a51d03 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:36.965244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf6efd-3682-444a-b16e-10399d149ca1 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.970390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff34e9-c939-42f8-acfb-ef47df63264d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.971106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff34e9-c939-42f8-acfb-ef47df63264d admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.971386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff34e9-c939-42f8-acfb-ef47df63264d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.971697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff34e9-c939-42f8-acfb-ef47df63264d admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:36.972342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff34e9-c939-42f8-acfb-ef47df63264d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:36.974321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff34e9-c939-42f8-acfb-ef47df63264d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:36.987409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fddd15-424e-4b1d-85ff-8f9c25a51d03 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:36.991935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18aef5a-f1db-44b3-968a-e6626a954e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:36.994182 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18aef5a-f1db-44b3-968a-e6626a954e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:36.994182 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18aef5a-f1db-44b3-968a-e6626a954e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:36.994182 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18aef5a-f1db-44b3-968a-e6626a954e18 admin admin] PATH_INFO: `/v3/projects/e54d379e790745f981d1724e9273fa06/users/2ee5334baae543ab95bb9a86c3ff92b5/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.014624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18aef5a-f1db-44b3-968a-e6626a954e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e54d379e790745f981d1724e9273fa06, user_id=2ee5334baae543ab95bb9a86c3ff92b5, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.022148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18aef5a-f1db-44b3-968a-e6626a954e18 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.142374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1178/2351] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:36 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:37.144468 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1174/2352] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:36 2022] PUT /identity/v3/projects/e54d379e790745f981d1724e9273fa06/users/2ee5334baae543ab95bb9a86c3ff92b5/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:37.146386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff34e9-c939-42f8-acfb-ef47df63264d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.159131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22451c59-c8a9-473e-8bdf-b0d4a0eae621 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.159847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22451c59-c8a9-473e-8bdf-b0d4a0eae621 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.160299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22451c59-c8a9-473e-8bdf-b0d4a0eae621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.194568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-22451c59-c8a9-473e-8bdf-b0d4a0eae621 None None] MFA Rules not processed for user `2ee5334baae543ab95bb9a86c3ff92b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:37.195907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-22451c59-c8a9-473e-8bdf-b0d4a0eae621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:37.219408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff34e9-c939-42f8-acfb-ef47df63264d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.225415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd755a-7876-4390-8b74-65bfc41d5cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.226285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd755a-7876-4390-8b74-65bfc41d5cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.226542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd755a-7876-4390-8b74-65bfc41d5cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.235655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd755a-7876-4390-8b74-65bfc41d5cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.235655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd755a-7876-4390-8b74-65bfc41d5cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.235655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd755a-7876-4390-8b74-65bfc41d5cd5 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.271324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1175/2353] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:37.276412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22451c59-c8a9-473e-8bdf-b0d4a0eae621 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.304139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22451c59-c8a9-473e-8bdf-b0d4a0eae621 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.310647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6c073-a462-47cb-b47d-f9c099b1eaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.311487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6c073-a462-47cb-b47d-f9c099b1eaaf admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.311803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6c073-a462-47cb-b47d-f9c099b1eaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.312127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6c073-a462-47cb-b47d-f9c099b1eaaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.312847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6c073-a462-47cb-b47d-f9c099b1eaaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.314805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6c073-a462-47cb-b47d-f9c099b1eaaf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.440497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1179/2354] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:37 2022] POST /identity/v3/users => generated 299 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:37.440497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd755a-7876-4390-8b74-65bfc41d5cd5 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.448992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd755a-7876-4390-8b74-65bfc41d5cd5 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.453591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342ffa0-9993-48d5-93d2-43b09405c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.454165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342ffa0-9993-48d5-93d2-43b09405c130 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.454448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342ffa0-9993-48d5-93d2-43b09405c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.455267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342ffa0-9993-48d5-93d2-43b09405c130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.455364 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342ffa0-9993-48d5-93d2-43b09405c130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.456944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342ffa0-9993-48d5-93d2-43b09405c130 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.470330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1180/2355] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:37.471508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342ffa0-9993-48d5-93d2-43b09405c130 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.482414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342ffa0-9993-48d5-93d2-43b09405c130 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.485771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1176/2356] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:37 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:37.489757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6c073-a462-47cb-b47d-f9c099b1eaaf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.542590 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f105f63-e2f5-4a9e-b908-36afa592ce3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.543253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f105f63-e2f5-4a9e-b908-36afa592ce3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.543529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f105f63-e2f5-4a9e-b908-36afa592ce3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.543828 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f105f63-e2f5-4a9e-b908-36afa592ce3c admin admin] PATH_INFO: `/v3/projects/e7d9adc99dea486bb99c2cc224ac3403/users/8b4f76d33b064563b61dfdcacf146cfa/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.555330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6c073-a462-47cb-b47d-f9c099b1eaaf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.560039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f105f63-e2f5-4a9e-b908-36afa592ce3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7d9adc99dea486bb99c2cc224ac3403, user_id=8b4f76d33b064563b61dfdcacf146cfa, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.560276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12655cc-a21e-442c-8e0f-76768883d53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.560890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12655cc-a21e-442c-8e0f-76768883d53f admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.561171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12655cc-a21e-442c-8e0f-76768883d53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.561446 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12655cc-a21e-442c-8e0f-76768883d53f admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.562117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12655cc-a21e-442c-8e0f-76768883d53f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.562767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f105f63-e2f5-4a9e-b908-36afa592ce3c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.564246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12655cc-a21e-442c-8e0f-76768883d53f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.638757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1181/2357] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:37 2022] PUT /identity/v3/projects/e7d9adc99dea486bb99c2cc224ac3403/users/8b4f76d33b064563b61dfdcacf146cfa/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:37.657369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1177/2358] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:37 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:37.657749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f105f63-e2f5-4a9e-b908-36afa592ce3c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.657943 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12655cc-a21e-442c-8e0f-76768883d53f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.657943 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12655cc-a21e-442c-8e0f-76768883d53f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.724107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773982c2-e587-4565-be62-4856f8f64e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.725657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773982c2-e587-4565-be62-4856f8f64e9c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.726212 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773982c2-e587-4565-be62-4856f8f64e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.726715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773982c2-e587-4565-be62-4856f8f64e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.727739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773982c2-e587-4565-be62-4856f8f64e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.729899 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f105f63-e2f5-4a9e-b908-36afa592ce3c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.730285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773982c2-e587-4565-be62-4856f8f64e9c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.735998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc68a42-d9d1-4072-b42a-d210350fd1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.736652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc68a42-d9d1-4072-b42a-d210350fd1ed admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.736905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc68a42-d9d1-4072-b42a-d210350fd1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.737172 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc68a42-d9d1-4072-b42a-d210350fd1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.738477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc68a42-d9d1-4072-b42a-d210350fd1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.740474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc68a42-d9d1-4072-b42a-d210350fd1ed admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.744755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1178/2359] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:37.748598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773982c2-e587-4565-be62-4856f8f64e9c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.760876 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773982c2-e587-4565-be62-4856f8f64e9c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.763988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1182/2360] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:37.764959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc68a42-d9d1-4072-b42a-d210350fd1ed admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.772427 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a45cd8-dfee-4d4e-8657-65f887dd5ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.773436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a45cd8-dfee-4d4e-8657-65f887dd5ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.773436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a45cd8-dfee-4d4e-8657-65f887dd5ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.774072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a45cd8-dfee-4d4e-8657-65f887dd5ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.774954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a45cd8-dfee-4d4e-8657-65f887dd5ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.776897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a45cd8-dfee-4d4e-8657-65f887dd5ccb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.777213 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc68a42-d9d1-4072-b42a-d210350fd1ed admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.783331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5818a3-3a8b-4b26-8aae-053976fec8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.784303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5818a3-3a8b-4b26-8aae-053976fec8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.784587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5818a3-3a8b-4b26-8aae-053976fec8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.784879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5818a3-3a8b-4b26-8aae-053976fec8bf admin admin] PATH_INFO: `/v3/projects/e7d9adc99dea486bb99c2cc224ac3403/users/8b4f76d33b064563b61dfdcacf146cfa/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.788323 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5818a3-3a8b-4b26-8aae-053976fec8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7d9adc99dea486bb99c2cc224ac3403, user_id=8b4f76d33b064563b61dfdcacf146cfa, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.791427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5818a3-3a8b-4b26-8aae-053976fec8bf admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.799160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1179/2361] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:37.802898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a45cd8-dfee-4d4e-8657-65f887dd5ccb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.805377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1183/2362] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:37 2022] PUT /identity/v3/projects/e7d9adc99dea486bb99c2cc224ac3403/users/8b4f76d33b064563b61dfdcacf146cfa/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:37.813386 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a45cd8-dfee-4d4e-8657-65f887dd5ccb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:37.816832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f8dca-2fbd-40a1-8737-9478cb8fda1b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.817114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f8dca-2fbd-40a1-8737-9478cb8fda1b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.817387 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f8dca-2fbd-40a1-8737-9478cb8fda1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.861874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-681f8dca-2fbd-40a1-8737-9478cb8fda1b None None] MFA Rules not processed for user `8b4f76d33b064563b61dfdcacf146cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:37.863114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-681f8dca-2fbd-40a1-8737-9478cb8fda1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:37.883874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4984e60-6c39-45f0-b68f-49d48ab440d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:37.884552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4984e60-6c39-45f0-b68f-49d48ab440d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:37.884840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4984e60-6c39-45f0-b68f-49d48ab440d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:37.885104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4984e60-6c39-45f0-b68f-49d48ab440d7 admin admin] PATH_INFO: `/v3/projects/63f1948133a348e8a0e46ea726cd9311/users/66700c5cf33d44668d495cf494fb40ff/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:37.909501 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4984e60-6c39-45f0-b68f-49d48ab440d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f1948133a348e8a0e46ea726cd9311, user_id=66700c5cf33d44668d495cf494fb40ff, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:37.914229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4984e60-6c39-45f0-b68f-49d48ab440d7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:37.960627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1184/2363] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:37.972160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f8dca-2fbd-40a1-8737-9478cb8fda1b None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:37.973552 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-681f8dca-2fbd-40a1-8737-9478cb8fda1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:38.035224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f8dca-2fbd-40a1-8737-9478cb8fda1b None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.044344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1180/2364] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:37 2022] PUT /identity/v3/projects/63f1948133a348e8a0e46ea726cd9311/users/66700c5cf33d44668d495cf494fb40ff/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:38.050763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a514d3e-7251-43d2-bad5-d6f1f56fe4f2 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.050763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a514d3e-7251-43d2-bad5-d6f1f56fe4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.050763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a514d3e-7251-43d2-bad5-d6f1f56fe4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.089491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7a514d3e-7251-43d2-bad5-d6f1f56fe4f2 None None] MFA Rules not processed for user `66700c5cf33d44668d495cf494fb40ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:38.090753 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7a514d3e-7251-43d2-bad5-d6f1f56fe4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:38.094113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd2354-a2e6-4b75-9897-99444af063c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.095071 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd2354-a2e6-4b75-9897-99444af063c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.095554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd2354-a2e6-4b75-9897-99444af063c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.096005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd2354-a2e6-4b75-9897-99444af063c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.098539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2dcd2354-a2e6-4b75-9897-99444af063c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:38.117580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:38.175450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1181/2365] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:38.178220 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd2354-a2e6-4b75-9897-99444af063c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.179100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd2354-a2e6-4b75-9897-99444af063c4 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.181750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a514d3e-7251-43d2-bad5-d6f1f56fe4f2 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:38.194398 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1185/2366] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:31:38.233514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:38.256812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a514d3e-7251-43d2-bad5-d6f1f56fe4f2 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.262337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c22a58-29df-4681-b867-a49a56e53177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.265524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c22a58-29df-4681-b867-a49a56e53177 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.265524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c22a58-29df-4681-b867-a49a56e53177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.265524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c22a58-29df-4681-b867-a49a56e53177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.265524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c22a58-29df-4681-b867-a49a56e53177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.265845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c22a58-29df-4681-b867-a49a56e53177 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.365541 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1182/2367] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:31:38 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:31:38.366889 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd2354-a2e6-4b75-9897-99444af063c4 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:38.379238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd2354-a2e6-4b75-9897-99444af063c4 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.452502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55375c4d-003b-4dae-bac1-ec6979ff2c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.453162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55375c4d-003b-4dae-bac1-ec6979ff2c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.453374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55375c4d-003b-4dae-bac1-ec6979ff2c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.453644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55375c4d-003b-4dae-bac1-ec6979ff2c97 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.454288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55375c4d-003b-4dae-bac1-ec6979ff2c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.456302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55375c4d-003b-4dae-bac1-ec6979ff2c97 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.597841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1186/2368] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:31:38 2022] POST /identity/v3/users => generated 295 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:31:38.598988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c22a58-29df-4681-b867-a49a56e53177 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:38.610002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c22a58-29df-4681-b867-a49a56e53177 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.614800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27771ad6-1e82-4489-992f-db552f745048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.616034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27771ad6-1e82-4489-992f-db552f745048 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.616537 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27771ad6-1e82-4489-992f-db552f745048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.616978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27771ad6-1e82-4489-992f-db552f745048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.618015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27771ad6-1e82-4489-992f-db552f745048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.619769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27771ad6-1e82-4489-992f-db552f745048 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.629959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1183/2369] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:31:38.634492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55375c4d-003b-4dae-bac1-ec6979ff2c97 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:38.644738 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55375c4d-003b-4dae-bac1-ec6979ff2c97 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.649516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92faaac-0105-4273-b00d-2f112d4d7e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.650171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92faaac-0105-4273-b00d-2f112d4d7e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.650447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92faaac-0105-4273-b00d-2f112d4d7e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.650718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92faaac-0105-4273-b00d-2f112d4d7e43 admin admin] PATH_INFO: `/v3/projects/85a3e1c01e7e420fad0d6a9a9c1cb577/users/325e1c1fc05448a2955989d4c28411e7/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.666901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92faaac-0105-4273-b00d-2f112d4d7e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a3e1c01e7e420fad0d6a9a9c1cb577, user_id=325e1c1fc05448a2955989d4c28411e7, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.669630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92faaac-0105-4273-b00d-2f112d4d7e43 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.787983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1187/2370] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:38 2022] PUT /identity/v3/projects/85a3e1c01e7e420fad0d6a9a9c1cb577/users/325e1c1fc05448a2955989d4c28411e7/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:38.796097 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27771ad6-1e82-4489-992f-db552f745048 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:38.804014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27771ad6-1e82-4489-992f-db552f745048 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.867248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ac804-4b80-4607-bf7a-de6feec84e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.867908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ac804-4b80-4607-bf7a-de6feec84e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.868186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ac804-4b80-4607-bf7a-de6feec84e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.868457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ac804-4b80-4607-bf7a-de6feec84e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.869103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ac804-4b80-4607-bf7a-de6feec84e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.870680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ac804-4b80-4607-bf7a-de6feec84e15 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.881090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1184/2371] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:31:38.885231 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92faaac-0105-4273-b00d-2f112d4d7e43 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:38.895742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92faaac-0105-4273-b00d-2f112d4d7e43 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:38.900693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0391c-c2f4-43e7-9963-4e79fe175f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:38.901353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0391c-c2f4-43e7-9963-4e79fe175f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.901637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0391c-c2f4-43e7-9963-4e79fe175f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.901916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0391c-c2f4-43e7-9963-4e79fe175f58 admin admin] PATH_INFO: `/v3/projects/85a3e1c01e7e420fad0d6a9a9c1cb577/users/325e1c1fc05448a2955989d4c28411e7/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.903905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0391c-c2f4-43e7-9963-4e79fe175f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a3e1c01e7e420fad0d6a9a9c1cb577, user_id=325e1c1fc05448a2955989d4c28411e7, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:38.906526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0391c-c2f4-43e7-9963-4e79fe175f58 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:38.918402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1188/2372] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:31:38 2022] PUT /identity/v3/projects/85a3e1c01e7e420fad0d6a9a9c1cb577/users/325e1c1fc05448a2955989d4c28411e7/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:31:38.919823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:38.930410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923db2ae-0371-4ddf-bea7-1e5871a1238e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:38.930696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923db2ae-0371-4ddf-bea7-1e5871a1238e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:38.931005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923db2ae-0371-4ddf-bea7-1e5871a1238e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:38.960249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-923db2ae-0371-4ddf-bea7-1e5871a1238e None None] MFA Rules not processed for user `325e1c1fc05448a2955989d4c28411e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:38.961443 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-923db2ae-0371-4ddf-bea7-1e5871a1238e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:39.086177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1185/2373] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:39.100887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0391c-c2f4-43e7-9963-4e79fe175f58 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:39.160453 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0391c-c2f4-43e7-9963-4e79fe175f58 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:39.164803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557dbf4-78e7-407a-be28-6c91e5cc4072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:39.165412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557dbf4-78e7-407a-be28-6c91e5cc4072 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:39.165694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557dbf4-78e7-407a-be28-6c91e5cc4072 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:39.165970 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557dbf4-78e7-407a-be28-6c91e5cc4072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:39.230628 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557dbf4-78e7-407a-be28-6c91e5cc4072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:39.232277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557dbf4-78e7-407a-be28-6c91e5cc4072 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:39.242100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1189/2374] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:39.528916 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:39.761005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:40.594327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923db2ae-0371-4ddf-bea7-1e5871a1238e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:40.595919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-923db2ae-0371-4ddf-bea7-1e5871a1238e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:40.618555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923db2ae-0371-4ddf-bea7-1e5871a1238e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:40.622522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495b551-842d-4248-8e35-5d6236f10ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:40.623135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495b551-842d-4248-8e35-5d6236f10ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:40.623470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495b551-842d-4248-8e35-5d6236f10ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:40.623784 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495b551-842d-4248-8e35-5d6236f10ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:40.736365 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495b551-842d-4248-8e35-5d6236f10ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:40.737814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495b551-842d-4248-8e35-5d6236f10ad9 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:40.756437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1186/2375] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:41.304926 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:41.518737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:41.686029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557dbf4-78e7-407a-be28-6c91e5cc4072 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:41.695347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557dbf4-78e7-407a-be28-6c91e5cc4072 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:41.699297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4a03a-4270-4a6a-8e94-73192209d9ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:41.700100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4a03a-4270-4a6a-8e94-73192209d9ac service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:41.700293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4a03a-4270-4a6a-8e94-73192209d9ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:41.700541 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4a03a-4270-4a6a-8e94-73192209d9ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:41.770693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4a03a-4270-4a6a-8e94-73192209d9ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:41.772257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4a03a-4270-4a6a-8e94-73192209d9ac service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:41.784202 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1190/2376] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:43.013161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64160d6f-f261-44f3-a941-fe2447ffac43 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:43.013745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64160d6f-f261-44f3-a941-fe2447ffac43 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:43.013745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64160d6f-f261-44f3-a941-fe2447ffac43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:43.033416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-64160d6f-f261-44f3-a941-fe2447ffac43 None None] MFA Rules not processed for user `f5543fbd902d441cb6f03d3155f1af8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:43.034656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-64160d6f-f261-44f3-a941-fe2447ffac43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:43.092007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1187/2377] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:43.175995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:43.290065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:44.206086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4a03a-4270-4a6a-8e94-73192209d9ac service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:44.215342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4a03a-4270-4a6a-8e94-73192209d9ac service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:44.219268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a4016-fae1-42e1-a1ed-7eadcd584c73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:44.219912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a4016-fae1-42e1-a1ed-7eadcd584c73 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:44.220192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a4016-fae1-42e1-a1ed-7eadcd584c73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:44.220465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a4016-fae1-42e1-a1ed-7eadcd584c73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:44.281948 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a4016-fae1-42e1-a1ed-7eadcd584c73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:44.283403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a4016-fae1-42e1-a1ed-7eadcd584c73 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:44.293469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1191/2378] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:45.574730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64160d6f-f261-44f3-a941-fe2447ffac43 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:45.590290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64160d6f-f261-44f3-a941-fe2447ffac43 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:45.594188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3e6d-bf7a-42bb-84a5-64f8879b4ea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:45.596558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3e6d-bf7a-42bb-84a5-64f8879b4ea5 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:45.596558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3e6d-bf7a-42bb-84a5-64f8879b4ea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:45.596558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3e6d-bf7a-42bb-84a5-64f8879b4ea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:45.605762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f3e6d-bf7a-42bb-84a5-64f8879b4ea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:45.607703 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f3e6d-bf7a-42bb-84a5-64f8879b4ea5 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:45.625223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1188/2379] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:45.733195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0b36-c948-4a05-9f5e-dcd48dd8f4fd None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:45.736118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0b36-c948-4a05-9f5e-dcd48dd8f4fd None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:45.736657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0b36-c948-4a05-9f5e-dcd48dd8f4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:45.792569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-a8bd0b36-c948-4a05-9f5e-dcd48dd8f4fd None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:45.794038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a8bd0b36-c948-4a05-9f5e-dcd48dd8f4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:45.980283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1192/2380] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:31:46.070848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3e6d-bf7a-42bb-84a5-64f8879b4ea5 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:46.072768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a50f3e6d-bf7a-42bb-84a5-64f8879b4ea5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:46.084600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:46.163912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:46.268036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3e6d-bf7a-42bb-84a5-64f8879b4ea5 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:46.279426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d386a-4f55-4358-b40d-0df849f3f1d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:46.280362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d386a-4f55-4358-b40d-0df849f3f1d8 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:46.280643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d386a-4f55-4358-b40d-0df849f3f1d8 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:46.280978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d386a-4f55-4358-b40d-0df849f3f1d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:46.300276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d386a-4f55-4358-b40d-0df849f3f1d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:46.302208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d386a-4f55-4358-b40d-0df849f3f1d8 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:46.341877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1189/2381] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:31:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:31:55.949256 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb438-95e0-47a7-b97c-de8e2b1eca5c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:55.949843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb438-95e0-47a7-b97c-de8e2b1eca5c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:55.949954 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb438-95e0-47a7-b97c-de8e2b1eca5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:55.982469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-0cbeb438-95e0-47a7-b97c-de8e2b1eca5c None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:55.984651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0cbeb438-95e0-47a7-b97c-de8e2b1eca5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:56.000895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1193/2382] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:56.017726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d386a-4f55-4358-b40d-0df849f3f1d8 service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:56.031639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d386a-4f55-4358-b40d-0df849f3f1d8 service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:56.037152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62677ea-2869-4e77-bdd3-eed27ed4f331 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:56.037854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62677ea-2869-4e77-bdd3-eed27ed4f331 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:56.038276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62677ea-2869-4e77-bdd3-eed27ed4f331 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:56.038643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62677ea-2869-4e77-bdd3-eed27ed4f331 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:56.050715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62677ea-2869-4e77-bdd3-eed27ed4f331 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:56.053062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62677ea-2869-4e77-bdd3-eed27ed4f331 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:56.066157 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1190/2383] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:31:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:31:56.873783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeb438-95e0-47a7-b97c-de8e2b1eca5c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:56.896134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=100 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:57.054642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeb438-95e0-47a7-b97c-de8e2b1eca5c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:57.065579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:57.066931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:57.068053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:57.069664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] PATH_INFO: `/v3/users/b3f78ffea4704c539d948e831f6a0b62` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:57.071445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f78ffea4704c539d948e831f6a0b62)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:57.075195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:57.386066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f78ffea4704c539d948e831f6a0b62'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:57.388840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f78ffea4704c539d948e831f6a0b62'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:57.500298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f78ffea4704c539d948e831f6a0b62'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:57.543099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f78ffea4704c539d948e831f6a0b62'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:57.558654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f78ffea4704c539d948e831f6a0b62'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:57.628086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1194/2384] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:56 2022] DELETE /identity/v3/users/b3f78ffea4704c539d948e831f6a0b62 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:57.634360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6370d-45fe-4719-95b1-1bb2ee8a26fa None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:57.634881 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6370d-45fe-4719-95b1-1bb2ee8a26fa None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:57.635211 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6370d-45fe-4719-95b1-1bb2ee8a26fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:57.692013 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-26b6370d-45fe-4719-95b1-1bb2ee8a26fa None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:57.693588 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-26b6370d-45fe-4719-95b1-1bb2ee8a26fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:57.868921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1191/2385] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:57.881391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:57.884186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:57.892340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6370d-45fe-4719-95b1-1bb2ee8a26fa None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:57.893843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-26b6370d-45fe-4719-95b1-1bb2ee8a26fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:57.917731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6370d-45fe-4719-95b1-1bb2ee8a26fa None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:57.942633 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:57.948936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:57.948936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:57.948936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] PATH_INFO: `/v3/users/2ee5334baae543ab95bb9a86c3ff92b5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:57.955018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee5334baae543ab95bb9a86c3ff92b5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:57.957097 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:58.119074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb280ac-ff87-491d-81d6-2325214acbf1 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:58.128721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedd8c8-dac0-413f-868b-2014a26c769c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:58.131499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedd8c8-dac0-413f-868b-2014a26c769c service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:58.134150 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedd8c8-dac0-413f-868b-2014a26c769c service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:58.136053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedd8c8-dac0-413f-868b-2014a26c769c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:58.157405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedd8c8-dac0-413f-868b-2014a26c769c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:58.162641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedd8c8-dac0-413f-868b-2014a26c769c service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:58.194822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1195/2386] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:31:58.277504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee5334baae543ab95bb9a86c3ff92b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:58.282761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee5334baae543ab95bb9a86c3ff92b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:58.606447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee5334baae543ab95bb9a86c3ff92b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:58.657749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee5334baae543ab95bb9a86c3ff92b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:58.665061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee5334baae543ab95bb9a86c3ff92b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:31:58.742918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1192/2387] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:31:57 2022] DELETE /identity/v3/users/2ee5334baae543ab95bb9a86c3ff92b5 => generated 0 bytes in 851 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:31:58.749357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ec7df-803c-41f0-86c5-4a62ee26cc1e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:58.755814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ec7df-803c-41f0-86c5-4a62ee26cc1e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:58.755814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ec7df-803c-41f0-86c5-4a62ee26cc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:58.810619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-732ec7df-803c-41f0-86c5-4a62ee26cc1e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:31:58.816808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-732ec7df-803c-41f0-86c5-4a62ee26cc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:58.918600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:59.043154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1196/2388] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:31:59.166155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:31:59.168776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:31:59.210995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:31:59.388582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dd42e-bea4-4a57-b0f3-72e9b81fa5ad admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:31:59.396562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cc617-1f5b-44d2-8b69-2e8f8bdbcbc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:31:59.396562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cc617-1f5b-44d2-8b69-2e8f8bdbcbc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:31:59.396562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cc617-1f5b-44d2-8b69-2e8f8bdbcbc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:31:59.396562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cc617-1f5b-44d2-8b69-2e8f8bdbcbc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:31:59.414719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41cc617-1f5b-44d2-8b69-2e8f8bdbcbc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:31:59.416644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41cc617-1f5b-44d2-8b69-2e8f8bdbcbc8 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:31:59.431626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1193/2389] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:32:00.099330 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ec7df-803c-41f0-86c5-4a62ee26cc1e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:00.101132 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-732ec7df-803c-41f0-86c5-4a62ee26cc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:00.115916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ec7df-803c-41f0-86c5-4a62ee26cc1e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:00.128011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:00.128746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:00.129125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:00.129477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] PATH_INFO: `/v3/users/4993091d3f504f88b25769cc34ab5623` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:00.132672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4993091d3f504f88b25769cc34ab5623)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:00.135193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:00.642190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cc617-1f5b-44d2-8b69-2e8f8bdbcbc8 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:00.644497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e41cc617-1f5b-44d2-8b69-2e8f8bdbcbc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:00.679613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cc617-1f5b-44d2-8b69-2e8f8bdbcbc8 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:00.696507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:00.698907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:00.699274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:00.699680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] PATH_INFO: `/v3/projects/60613c7205f34175980c3823e6cd5baa` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:00.702417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] RBAC: Authorizing `identity:delete_project(project_id=60613c7205f34175980c3823e6cd5baa)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:00.706492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:00.775898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4993091d3f504f88b25769cc34ab5623'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:00.777639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4993091d3f504f88b25769cc34ab5623'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:00.889548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4993091d3f504f88b25769cc34ab5623'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:00.930878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4993091d3f504f88b25769cc34ab5623'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:00.932484 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-fb6494d4-b40f-4695-8f9b-531f4400e8bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4993091d3f504f88b25769cc34ab5623'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:00.960811 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60613c7205f34175980c3823e6cd5baa', 'request_id': 'req-826dbfdf-9f33-4e24-84b7-708dd92f183d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:00.982234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60613c7205f34175980c3823e6cd5baa', 'request_id': 'req-826dbfdf-9f33-4e24-84b7-708dd92f183d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:00.997153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1197/2390] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:32:00 2022] DELETE /identity/v3/users/4993091d3f504f88b25769cc34ab5623 => generated 0 bytes in 896 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:01.002204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2eb4d6-a67d-449f-b88c-db60eae49911 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:01.003280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2eb4d6-a67d-449f-b88c-db60eae49911 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:01.006069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2eb4d6-a67d-449f-b88c-db60eae49911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:01.058277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-cf2eb4d6-a67d-449f-b88c-db60eae49911 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:01.059798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cf2eb4d6-a67d-449f-b88c-db60eae49911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:01.116920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60613c7205f34175980c3823e6cd5baa', 'request_id': 'req-826dbfdf-9f33-4e24-84b7-708dd92f183d'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.180093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] The token cache is being invalidate because project 60613c7205f34175980c3823e6cd5baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:01.180093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.180093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.180093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1194/2391] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:00 2022] DELETE /identity/v3/projects/60613c7205f34175980c3823e6cd5baa => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:01.180093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:01.183680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:01.305410 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1198/2392] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:01.306849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2eb4d6-a67d-449f-b88c-db60eae49911 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:01.309578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cf2eb4d6-a67d-449f-b88c-db60eae49911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:01.325422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2eb4d6-a67d-449f-b88c-db60eae49911 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:01.337292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:01.339198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:01.340165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:01.340507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] PATH_INFO: `/v3/projects/e54d379e790745f981d1724e9273fa06` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:01.344360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e54d379e790745f981d1724e9273fa06)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:01.349083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:01.412940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dbfdf-9f33-4e24-84b7-708dd92f183d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:01.424204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:01.424933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:01.425909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:01.426189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] PATH_INFO: `/v3/users/e18d3231ba03446cac59dcbd1a512628` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:01.429993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18d3231ba03446cac59dcbd1a512628)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:01.433344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:01.542286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:01.564309 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e54d379e790745f981d1724e9273fa06', 'request_id': 'req-5490402c-60bb-45c4-a961-c9fb17f70f98'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.576019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e54d379e790745f981d1724e9273fa06', 'request_id': 'req-5490402c-60bb-45c4-a961-c9fb17f70f98'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.675609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e54d379e790745f981d1724e9273fa06', 'request_id': 'req-5490402c-60bb-45c4-a961-c9fb17f70f98'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.684761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:01.703097 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18d3231ba03446cac59dcbd1a512628'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.705130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18d3231ba03446cac59dcbd1a512628'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.707000 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] The token cache is being invalidate because project e54d379e790745f981d1724e9273fa06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:01.712528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.714527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.728083 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1199/2393] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:01 2022] DELETE /identity/v3/projects/e54d379e790745f981d1724e9273fa06 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:01.729272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:01.735193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:01.799657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18d3231ba03446cac59dcbd1a512628'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.869278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18d3231ba03446cac59dcbd1a512628'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.873207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18d3231ba03446cac59dcbd1a512628'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:01.929773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1195/2394] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:32:01 2022] DELETE /identity/v3/users/e18d3231ba03446cac59dcbd1a512628 => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:01.931075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:01.932892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:02.000294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490402c-60bb-45c4-a961-c9fb17f70f98 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:02.122556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57cf67-95ab-4dfc-a11b-6f32d8d73ff3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:02.131595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64f5a0-fe0b-4c9d-b36c-a3117186ccdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:02.132433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64f5a0-fe0b-4c9d-b36c-a3117186ccdc service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:02.132819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64f5a0-fe0b-4c9d-b36c-a3117186ccdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:02.133125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64f5a0-fe0b-4c9d-b36c-a3117186ccdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:02.134557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:02.135421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-eb64f5a0-fe0b-4c9d-b36c-a3117186ccdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:02.136240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:02.136528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:02.136805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] PATH_INFO: `/v3/users/66700c5cf33d44668d495cf494fb40ff` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:02.140751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66700c5cf33d44668d495cf494fb40ff)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:02.149521 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:02.174160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64f5a0-fe0b-4c9d-b36c-a3117186ccdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:02.179552 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64f5a0-fe0b-4c9d-b36c-a3117186ccdc service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:02.212380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1200/2395] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:02.499860 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66700c5cf33d44668d495cf494fb40ff'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:02.500518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66700c5cf33d44668d495cf494fb40ff'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:02.614128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66700c5cf33d44668d495cf494fb40ff'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:02.656165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66700c5cf33d44668d495cf494fb40ff'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:02.659052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66700c5cf33d44668d495cf494fb40ff'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:02.689344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1196/2396] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:32:01 2022] DELETE /identity/v3/users/66700c5cf33d44668d495cf494fb40ff => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:02.961846 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64f5a0-fe0b-4c9d-b36c-a3117186ccdc service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:02.965121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-eb64f5a0-fe0b-4c9d-b36c-a3117186ccdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:02.998682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64f5a0-fe0b-4c9d-b36c-a3117186ccdc service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:03.142020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:03.143737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:03.214412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:03.215181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:03.215431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:03.215688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] PATH_INFO: `/v3/projects/9fe7d55243ed4fb395edbda2ee620f35` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:03.218011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fe7d55243ed4fb395edbda2ee620f35)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:03.220289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:03.263343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:03.332948 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d1c10-85b0-4beb-bf90-30cd0fe32ce3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:03.340098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:03.341756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:03.341756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:03.341756 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] PATH_INFO: `/v3/projects/2fc699337d0b407aa2aceebcf116671e` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:03.343668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fc699337d0b407aa2aceebcf116671e)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:03.347040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:03.367726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:03.416243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fe7d55243ed4fb395edbda2ee620f35', 'request_id': 'req-61361031-b9db-41d2-9a6e-e812ba16413d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:03.431192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fe7d55243ed4fb395edbda2ee620f35', 'request_id': 'req-61361031-b9db-41d2-9a6e-e812ba16413d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:03.557409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fc699337d0b407aa2aceebcf116671e', 'request_id': 'req-1025ca41-49f4-4ced-870c-a27a67333544'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:03.585184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fc699337d0b407aa2aceebcf116671e', 'request_id': 'req-1025ca41-49f4-4ced-870c-a27a67333544'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:03.595676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fe7d55243ed4fb395edbda2ee620f35', 'request_id': 'req-61361031-b9db-41d2-9a6e-e812ba16413d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:03.616637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] The token cache is being invalidate because project 9fe7d55243ed4fb395edbda2ee620f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:03.616637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:03.633160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:03.633160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1201/2397] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:02 2022] DELETE /identity/v3/projects/9fe7d55243ed4fb395edbda2ee620f35 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:03.710193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fc699337d0b407aa2aceebcf116671e', 'request_id': 'req-1025ca41-49f4-4ced-870c-a27a67333544'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:03.739257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] The token cache is being invalidate because project 2fc699337d0b407aa2aceebcf116671e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:03.739865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:03.742342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:03.748660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1197/2398] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:03 2022] DELETE /identity/v3/projects/2fc699337d0b407aa2aceebcf116671e => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:03.750215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:03.753721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:03.888100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61361031-b9db-41d2-9a6e-e812ba16413d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:03.925328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:03.927437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:04.021088 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:04.022776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:04.022876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:04.023263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] PATH_INFO: `/v3/users/8b4f76d33b064563b61dfdcacf146cfa` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:04.024577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4f76d33b064563b61dfdcacf146cfa)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:04.028185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:04.049237 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025ca41-49f4-4ced-870c-a27a67333544 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:04.056837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:04.057508 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:04.057788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:04.058144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] PATH_INFO: `/v3/projects/63f1948133a348e8a0e46ea726cd9311` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:04.059293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f1948133a348e8a0e46ea726cd9311)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:04.061289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:04.314135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f1948133a348e8a0e46ea726cd9311', 'request_id': 'req-ec0dab11-b477-4e19-86c0-82675cc6df3b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.318848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4f76d33b064563b61dfdcacf146cfa'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.321920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4f76d33b064563b61dfdcacf146cfa'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.322804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f1948133a348e8a0e46ea726cd9311', 'request_id': 'req-ec0dab11-b477-4e19-86c0-82675cc6df3b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.417368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f1948133a348e8a0e46ea726cd9311', 'request_id': 'req-ec0dab11-b477-4e19-86c0-82675cc6df3b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.420470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:04.435075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] The token cache is being invalidate because project 63f1948133a348e8a0e46ea726cd9311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:04.435847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.436757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.438639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1198/2399] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:03 2022] DELETE /identity/v3/projects/63f1948133a348e8a0e46ea726cd9311 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:04.443079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:04.445599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:04.490804 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4f76d33b064563b61dfdcacf146cfa'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.508400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:04.520826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4f76d33b064563b61dfdcacf146cfa'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.527487 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4f76d33b064563b61dfdcacf146cfa'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.562381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dab11-b477-4e19-86c0-82675cc6df3b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:04.565114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1202/2400] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:32:03 2022] DELETE /identity/v3/users/8b4f76d33b064563b61dfdcacf146cfa => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:04.689764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:04.690915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:04.691328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:04.692007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] PATH_INFO: `/v3/users/325e1c1fc05448a2955989d4c28411e7` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:04.698878 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=325e1c1fc05448a2955989d4c28411e7)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:04.702152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:04.927081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325e1c1fc05448a2955989d4c28411e7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:04.928863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325e1c1fc05448a2955989d4c28411e7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:05.021976 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325e1c1fc05448a2955989d4c28411e7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:05.038333 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325e1c1fc05448a2955989d4c28411e7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:05.039207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e1944667-5354-4b7e-b824-a8c1026ce260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325e1c1fc05448a2955989d4c28411e7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:05.057156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1199/2401] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:32:04 2022] DELETE /identity/v3/users/325e1c1fc05448a2955989d4c28411e7 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:05.203315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:05.215689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:05.236446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6d6dc-bfab-4c21-94dd-eeb6042d4a1c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:05.480811 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:05.481548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:05.482672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:05.482672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] PATH_INFO: `/v3/projects/e7d9adc99dea486bb99c2cc224ac3403` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:05.486916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7d9adc99dea486bb99c2cc224ac3403)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:05.489902 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:05.640048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7d9adc99dea486bb99c2cc224ac3403', 'request_id': 'req-87b41437-2dbb-4d9e-ad47-d89d2f495bff'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:05.648542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7d9adc99dea486bb99c2cc224ac3403', 'request_id': 'req-87b41437-2dbb-4d9e-ad47-d89d2f495bff'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:05.743207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7d9adc99dea486bb99c2cc224ac3403', 'request_id': 'req-87b41437-2dbb-4d9e-ad47-d89d2f495bff'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:05.760896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] The token cache is being invalidate because project e7d9adc99dea486bb99c2cc224ac3403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:05.761395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:05.762082 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:05.768639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68374d2-83ae-4acc-a2ae-46c5ce7cc25f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:05.768948 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68374d2-83ae-4acc-a2ae-46c5ce7cc25f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:05.770998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68374d2-83ae-4acc-a2ae-46c5ce7cc25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:05.779645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1203/2402] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:05 2022] DELETE /identity/v3/projects/e7d9adc99dea486bb99c2cc224ac3403 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:05.801432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b68374d2-83ae-4acc-a2ae-46c5ce7cc25f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:05.802905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b68374d2-83ae-4acc-a2ae-46c5ce7cc25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:05.829439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:05.834711 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:05.836407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:05.887789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1200/2403] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:05.900356 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b043fe3-ea60-4de9-b077-274a802c116f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:05.901262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b043fe3-ea60-4de9-b077-274a802c116f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:05.901262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b043fe3-ea60-4de9-b077-274a802c116f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:05.950220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6b043fe3-ea60-4de9-b077-274a802c116f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:05.952417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6b043fe3-ea60-4de9-b077-274a802c116f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:05.971123 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1201/2404] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:05.976728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b043fe3-ea60-4de9-b077-274a802c116f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:05.991696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b043fe3-ea60-4de9-b077-274a802c116f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:06.012235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c908d05-4371-40a1-9fb7-32a03eca5bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:06.013886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c908d05-4371-40a1-9fb7-32a03eca5bbe admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.013886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c908d05-4371-40a1-9fb7-32a03eca5bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.013886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c908d05-4371-40a1-9fb7-32a03eca5bbe admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.014355 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c908d05-4371-40a1-9fb7-32a03eca5bbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:32:06.014445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c908d05-4371-40a1-9fb7-32a03eca5bbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:06.016106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c908d05-4371-40a1-9fb7-32a03eca5bbe admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:06.040695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b41437-2dbb-4d9e-ad47-d89d2f495bff admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:06.060045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1202/2405] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:32:06.065570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c908d05-4371-40a1-9fb7-32a03eca5bbe admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:06.065708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:06.065708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.065708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.065708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] PATH_INFO: `/v3/projects/85a3e1c01e7e420fad0d6a9a9c1cb577` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.066980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a3e1c01e7e420fad0d6a9a9c1cb577)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:06.069843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:06.084758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c908d05-4371-40a1-9fb7-32a03eca5bbe admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:06.098349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6772f4e-017e-41be-b49f-385ee7a60dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:06.099702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6772f4e-017e-41be-b49f-385ee7a60dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.100138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6772f4e-017e-41be-b49f-385ee7a60dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.100522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6772f4e-017e-41be-b49f-385ee7a60dc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.101758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:06.102010 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6772f4e-017e-41be-b49f-385ee7a60dc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:06.105572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6772f4e-017e-41be-b49f-385ee7a60dc6 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:06.250841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a3e1c01e7e420fad0d6a9a9c1cb577', 'request_id': 'req-0aeb9415-ceda-42ee-85f9-134b154aabd8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:06.258911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a3e1c01e7e420fad0d6a9a9c1cb577', 'request_id': 'req-0aeb9415-ceda-42ee-85f9-134b154aabd8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:06.262128 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1203/2406] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:32:06 2022] POST /identity/v3/projects => generated 375 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:32:06.263060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6772f4e-017e-41be-b49f-385ee7a60dc6 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:06.332017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a3e1c01e7e420fad0d6a9a9c1cb577', 'request_id': 'req-0aeb9415-ceda-42ee-85f9-134b154aabd8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:06.350634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] The token cache is being invalidate because project 85a3e1c01e7e420fad0d6a9a9c1cb577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:06.350992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:06.351949 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-0aeb9415-ceda-42ee-85f9-134b154aabd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:06.352860 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6772f4e-017e-41be-b49f-385ee7a60dc6 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:06.357578 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1204/2407] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:05 2022] DELETE /identity/v3/projects/85a3e1c01e7e420fad0d6a9a9c1cb577 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:32:06.359090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ff010-3709-4235-87c9-70eae8581114 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.359374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ff010-3709-4235-87c9-70eae8581114 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.359681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ff010-3709-4235-87c9-70eae8581114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.365522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62342fe1-a498-4ed4-ae10-aa3de0eaa772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:06.366193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62342fe1-a498-4ed4-ae10-aa3de0eaa772 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.366474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62342fe1-a498-4ed4-ae10-aa3de0eaa772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.366750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62342fe1-a498-4ed4-ae10-aa3de0eaa772 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.369150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62342fe1-a498-4ed4-ae10-aa3de0eaa772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:06.369919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62342fe1-a498-4ed4-ae10-aa3de0eaa772 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:06.389585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-204ff010-3709-4235-87c9-70eae8581114 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:06.391200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-204ff010-3709-4235-87c9-70eae8581114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:06.412324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1205/2408] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:06.421794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696eb8b8-c8e5-42da-b819-cade205c3432 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.422106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696eb8b8-c8e5-42da-b819-cade205c3432 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.422346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696eb8b8-c8e5-42da-b819-cade205c3432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.445626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1204/2409] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:32:06 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:32:06.451273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62342fe1-a498-4ed4-ae10-aa3de0eaa772 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:06.451481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-696eb8b8-c8e5-42da-b819-cade205c3432 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:06.452751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-696eb8b8-c8e5-42da-b819-cade205c3432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:06.453293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-62342fe1-a498-4ed4-ae10-aa3de0eaa772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:06.466701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1206/2410] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:32:06.473028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62342fe1-a498-4ed4-ae10-aa3de0eaa772 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:06.473610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696eb8b8-c8e5-42da-b819-cade205c3432 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:06.478089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b812edc-5b2d-45ac-b5e7-321bbaf60b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:06.478794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b812edc-5b2d-45ac-b5e7-321bbaf60b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.479137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b812edc-5b2d-45ac-b5e7-321bbaf60b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.479477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b812edc-5b2d-45ac-b5e7-321bbaf60b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.480495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b812edc-5b2d-45ac-b5e7-321bbaf60b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:06.482154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b812edc-5b2d-45ac-b5e7-321bbaf60b08 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:06.487155 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696eb8b8-c8e5-42da-b819-cade205c3432 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:06.493599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e1402-9304-4773-86be-36f27b43665c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:06.494538 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e1402-9304-4773-86be-36f27b43665c admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.494859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e1402-9304-4773-86be-36f27b43665c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.495115 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e1402-9304-4773-86be-36f27b43665c admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.495820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e1402-9304-4773-86be-36f27b43665c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:32:06.496151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e1402-9304-4773-86be-36f27b43665c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:06.497213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1205/2411] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:32:06.498068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e1402-9304-4773-86be-36f27b43665c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:06.500940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b812edc-5b2d-45ac-b5e7-321bbaf60b08 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:06.512025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b812edc-5b2d-45ac-b5e7-321bbaf60b08 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:06.514370 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1207/2412] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:32:06.521273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcd6bf-ede8-4ecf-a8e7-8288a584284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:06.525845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcd6bf-ede8-4ecf-a8e7-8288a584284a admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.525845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcd6bf-ede8-4ecf-a8e7-8288a584284a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.525845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcd6bf-ede8-4ecf-a8e7-8288a584284a admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.525845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bcd6bf-ede8-4ecf-a8e7-8288a584284a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:06.525845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bcd6bf-ede8-4ecf-a8e7-8288a584284a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:06.527023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e1402-9304-4773-86be-36f27b43665c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:06.538070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1206/2413] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:32:06.542026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcd6bf-ede8-4ecf-a8e7-8288a584284a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:06.543192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e1402-9304-4773-86be-36f27b43665c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:06.553347 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969a2e3-4a92-4871-8d0c-65a4aebc09af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:06.556098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969a2e3-4a92-4871-8d0c-65a4aebc09af admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.556098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969a2e3-4a92-4871-8d0c-65a4aebc09af admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.556098 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969a2e3-4a92-4871-8d0c-65a4aebc09af admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.556433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969a2e3-4a92-4871-8d0c-65a4aebc09af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:06.558795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969a2e3-4a92-4871-8d0c-65a4aebc09af admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:06.561842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcd6bf-ede8-4ecf-a8e7-8288a584284a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:06.567194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ce2de-3222-4de0-9f14-304c57e61de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:06.568015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ce2de-3222-4de0-9f14-304c57e61de3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.568839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ce2de-3222-4de0-9f14-304c57e61de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.569223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ce2de-3222-4de0-9f14-304c57e61de3 admin admin] PATH_INFO: `/v3/projects/29ac989524a34d65bcefc990244d02dd/users/d86ccae798764807976ab6c52b5f50a7/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.592843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ce2de-3222-4de0-9f14-304c57e61de3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ac989524a34d65bcefc990244d02dd, user_id=d86ccae798764807976ab6c52b5f50a7, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:06.595554 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ce2de-3222-4de0-9f14-304c57e61de3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:06.668909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1208/2414] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:32:06 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:32:06.675775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969a2e3-4a92-4871-8d0c-65a4aebc09af admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:06.683518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1207/2415] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:32:06 2022] PUT /identity/v3/projects/29ac989524a34d65bcefc990244d02dd/users/d86ccae798764807976ab6c52b5f50a7/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:06.771880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bddc12-edb2-4c81-81c7-bf5d6b3d0fe6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.772275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bddc12-edb2-4c81-81c7-bf5d6b3d0fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.772809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bddc12-edb2-4c81-81c7-bf5d6b3d0fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.821877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969a2e3-4a92-4871-8d0c-65a4aebc09af admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:06.840589 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-07bddc12-edb2-4c81-81c7-bf5d6b3d0fe6 None None] MFA Rules not processed for user `d86ccae798764807976ab6c52b5f50a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:06.841978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-07bddc12-edb2-4c81-81c7-bf5d6b3d0fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:06.861709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:06.930728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844216a-9909-47db-814b-ee82528ef2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:06.931872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844216a-9909-47db-814b-ee82528ef2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:06.932452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844216a-9909-47db-814b-ee82528ef2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:06.932731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844216a-9909-47db-814b-ee82528ef2b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:06.933372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844216a-9909-47db-814b-ee82528ef2b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:06.944006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844216a-9909-47db-814b-ee82528ef2b6 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:06.953353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:06.968185 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1208/2416] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:32:06.992439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bddc12-edb2-4c81-81c7-bf5d6b3d0fe6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:06.992439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-07bddc12-edb2-4c81-81c7-bf5d6b3d0fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:07.070878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1209/2417] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:32:06 2022] POST /identity/v3/users => generated 293 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:32:07.070878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844216a-9909-47db-814b-ee82528ef2b6 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:07.084923 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bddc12-edb2-4c81-81c7-bf5d6b3d0fe6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:07.085907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844216a-9909-47db-814b-ee82528ef2b6 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:07.094051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aff13f-daa2-489c-a35e-7d37d8f6e44d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:07.094693 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aff13f-daa2-489c-a35e-7d37d8f6e44d service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:07.094988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aff13f-daa2-489c-a35e-7d37d8f6e44d service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:07.095248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aff13f-daa2-489c-a35e-7d37d8f6e44d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:07.100404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab49740-efc6-4057-ba1e-a80f9ca980cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:07.106230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab49740-efc6-4057-ba1e-a80f9ca980cb admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:07.106230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab49740-efc6-4057-ba1e-a80f9ca980cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:07.106230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab49740-efc6-4057-ba1e-a80f9ca980cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:07.106230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab49740-efc6-4057-ba1e-a80f9ca980cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:07.106230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab49740-efc6-4057-ba1e-a80f9ca980cb admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:07.123102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aff13f-daa2-489c-a35e-7d37d8f6e44d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:07.124694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aff13f-daa2-489c-a35e-7d37d8f6e44d service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:07.144119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1210/2418] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:32:07.145514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab49740-efc6-4057-ba1e-a80f9ca980cb admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:07.157384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1209/2419] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:32:07.164532 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab49740-efc6-4057-ba1e-a80f9ca980cb admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:07.169618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a22d6-542b-4a74-bb76-4c7698a4941e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:07.170239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a22d6-542b-4a74-bb76-4c7698a4941e admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:07.170632 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a22d6-542b-4a74-bb76-4c7698a4941e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:07.170919 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a22d6-542b-4a74-bb76-4c7698a4941e admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:07.171597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a22d6-542b-4a74-bb76-4c7698a4941e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:07.173281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a22d6-542b-4a74-bb76-4c7698a4941e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:07.191641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1211/2420] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:32:07.195254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aff13f-daa2-489c-a35e-7d37d8f6e44d service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:07.215253 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aff13f-daa2-489c-a35e-7d37d8f6e44d service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:07.226480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425f160-310e-43cc-a049-9271fbd5d6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:07.227229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425f160-310e-43cc-a049-9271fbd5d6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:07.227513 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425f160-310e-43cc-a049-9271fbd5d6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:07.227816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425f160-310e-43cc-a049-9271fbd5d6fe admin admin] PATH_INFO: `/v3/projects/7c415434529f46e88633252dd5c4c195/users/29816ad3d6854dd6aee9233fc7f97dc6/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:07.247011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425f160-310e-43cc-a049-9271fbd5d6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c415434529f46e88633252dd5c4c195, user_id=29816ad3d6854dd6aee9233fc7f97dc6, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:07.251321 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425f160-310e-43cc-a049-9271fbd5d6fe admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:07.388279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1210/2421] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:32:07 2022] PUT /identity/v3/projects/7c415434529f46e88633252dd5c4c195/users/29816ad3d6854dd6aee9233fc7f97dc6/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:07.400162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471754f4-6bdb-43af-acea-427b13653795 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:07.400693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471754f4-6bdb-43af-acea-427b13653795 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:07.400870 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471754f4-6bdb-43af-acea-427b13653795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:07.446237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-471754f4-6bdb-43af-acea-427b13653795 None None] MFA Rules not processed for user `29816ad3d6854dd6aee9233fc7f97dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:07.447486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-471754f4-6bdb-43af-acea-427b13653795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:07.557178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1212/2422] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:07.570534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425f160-310e-43cc-a049-9271fbd5d6fe admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:07.572579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5425f160-310e-43cc-a049-9271fbd5d6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:07.674302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425f160-310e-43cc-a049-9271fbd5d6fe admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:07.678193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552b6d0-1fcd-43c5-aece-64bce0eb73a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:07.678818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b6d0-1fcd-43c5-aece-64bce0eb73a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:07.679304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b6d0-1fcd-43c5-aece-64bce0eb73a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:07.679778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b6d0-1fcd-43c5-aece-64bce0eb73a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:07.693278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552b6d0-1fcd-43c5-aece-64bce0eb73a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:07.694810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552b6d0-1fcd-43c5-aece-64bce0eb73a6 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:07.708351 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1211/2423] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:09.255589 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48e26-51f5-467b-8ec1-28eae1c7e051 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:09.258793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48e26-51f5-467b-8ec1-28eae1c7e051 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:09.263837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48e26-51f5-467b-8ec1-28eae1c7e051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:09.291126 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-74f48e26-51f5-467b-8ec1-28eae1c7e051 None None] MFA Rules not processed for user `d86ccae798764807976ab6c52b5f50a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:09.305206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-74f48e26-51f5-467b-8ec1-28eae1c7e051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:09.402507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1213/2424] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:11.328478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552b6d0-1fcd-43c5-aece-64bce0eb73a6 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:11.447873 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:11.487985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552b6d0-1fcd-43c5-aece-64bce0eb73a6 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:11.496973 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:11.497648 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.499247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.499555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] PATH_INFO: `/v3/users/29816ad3d6854dd6aee9233fc7f97dc6` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.503948 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29816ad3d6854dd6aee9233fc7f97dc6)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:11.510755 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:11.753020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29816ad3d6854dd6aee9233fc7f97dc6'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.753346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29816ad3d6854dd6aee9233fc7f97dc6'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.863368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29816ad3d6854dd6aee9233fc7f97dc6'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.896525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29816ad3d6854dd6aee9233fc7f97dc6'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.896736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29816ad3d6854dd6aee9233fc7f97dc6'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:11.918716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1212/2425] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:32:11 2022] DELETE /identity/v3/users/29816ad3d6854dd6aee9233fc7f97dc6 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:11.925505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bb765-4a88-4e48-a10f-bcdf2d577294 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:11.926094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bb765-4a88-4e48-a10f-bcdf2d577294 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:11.929005 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bb765-4a88-4e48-a10f-bcdf2d577294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:11.960456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-a72bb765-4a88-4e48-a10f-bcdf2d577294 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:11.963647 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a72bb765-4a88-4e48-a10f-bcdf2d577294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:12.134163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1214/2426] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:12.242999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:12.244708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:12.415147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221e1b3-b593-4189-a6e6-fc0011aa4119 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:12.432502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d348d-be2d-4247-81c7-f66429d181cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:12.433196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d348d-be2d-4247-81c7-f66429d181cf service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:12.433634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d348d-be2d-4247-81c7-f66429d181cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:12.434082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d348d-be2d-4247-81c7-f66429d181cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:12.466239 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d348d-be2d-4247-81c7-f66429d181cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:12.469436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d348d-be2d-4247-81c7-f66429d181cf service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:12.504805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1213/2427] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:13.390271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bb765-4a88-4e48-a10f-bcdf2d577294 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:13.393201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a72bb765-4a88-4e48-a10f-bcdf2d577294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:13.406101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:13.414450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bb765-4a88-4e48-a10f-bcdf2d577294 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:13.422478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:13.423174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:13.423427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:13.423759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] PATH_INFO: `/v3/projects/7c415434529f46e88633252dd5c4c195` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:13.439670 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c415434529f46e88633252dd5c4c195)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:13.441671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:13.659011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c415434529f46e88633252dd5c4c195', 'request_id': 'req-079ea38d-0096-4cbd-af85-34517201b87e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:13.719180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c415434529f46e88633252dd5c4c195', 'request_id': 'req-079ea38d-0096-4cbd-af85-34517201b87e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:13.847975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c415434529f46e88633252dd5c4c195', 'request_id': 'req-079ea38d-0096-4cbd-af85-34517201b87e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:13.850171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] The token cache is being invalidate because project 7c415434529f46e88633252dd5c4c195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:13.850664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:13.852170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-079ea38d-0096-4cbd-af85-34517201b87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:13.857893 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1215/2428] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:13 2022] DELETE /identity/v3/projects/7c415434529f46e88633252dd5c4c195 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:18.595987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d348d-be2d-4247-81c7-f66429d181cf service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:18.598073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-336d348d-be2d-4247-81c7-f66429d181cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:18.616524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d348d-be2d-4247-81c7-f66429d181cf service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:18.695229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d972f7-c355-496a-8941-4e1d535eb60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:18.695883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d972f7-c355-496a-8941-4e1d535eb60a service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:18.696184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d972f7-c355-496a-8941-4e1d535eb60a service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:18.696470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d972f7-c355-496a-8941-4e1d535eb60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:18.698428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b2d972f7-c355-496a-8941-4e1d535eb60a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:18.711189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:18.838867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d972f7-c355-496a-8941-4e1d535eb60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:18.840446 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d972f7-c355-496a-8941-4e1d535eb60a service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:18.853357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1214/2429] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:19.482116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0a5ff-eddc-4472-a95c-5141b1fadd35 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:19.482654 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0a5ff-eddc-4472-a95c-5141b1fadd35 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:19.482876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0a5ff-eddc-4472-a95c-5141b1fadd35 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:19.485350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1216/2430] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:32:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:32:19.493618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1be1c4-00ad-4d52-b912-7f4e7367da82 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:19.494044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1be1c4-00ad-4d52-b912-7f4e7367da82 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:19.494622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1be1c4-00ad-4d52-b912-7f4e7367da82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:19.524343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6b1be1c4-00ad-4d52-b912-7f4e7367da82 None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:19.525954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6b1be1c4-00ad-4d52-b912-7f4e7367da82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:19.621059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1215/2431] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:19.649510 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0a5ff-eddc-4472-a95c-5141b1fadd35 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:19.649510 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-39e0a5ff-eddc-4472-a95c-5141b1fadd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:19.725697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0a5ff-eddc-4472-a95c-5141b1fadd35 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:19.733125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5505723-8d4a-4e94-a57f-fa8f8f5642b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '237bbeebe2a949aba444353c1d2ee04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:19.733745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5505723-8d4a-4e94-a57f-fa8f8f5642b8 service glance] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:19.734026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5505723-8d4a-4e94-a57f-fa8f8f5642b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:19.734302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5505723-8d4a-4e94-a57f-fa8f8f5642b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:19.746520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5505723-8d4a-4e94-a57f-fa8f8f5642b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:19.748321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5505723-8d4a-4e94-a57f-fa8f8f5642b8 service glance] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:19.760474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1217/2432] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:20.013752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca184c-da8a-457e-bd84-688095221bdf None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:20.014364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca184c-da8a-457e-bd84-688095221bdf None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:20.014868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca184c-da8a-457e-bd84-688095221bdf None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:20.018060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1216/2433] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:32:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:32:20.026288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9dee8-ff56-4bab-9bd7-ac068580b55c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:20.026989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9dee8-ff56-4bab-9bd7-ac068580b55c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:20.027266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9dee8-ff56-4bab-9bd7-ac068580b55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:20.059302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-5ba9dee8-ff56-4bab-9bd7-ac068580b55c None None] MFA Rules not processed for user `9a89a2e6aeb4495a84a61bd82b02bfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:20.061194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5ba9dee8-ff56-4bab-9bd7-ac068580b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:20.091739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1218/2434] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:20.103982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca184c-da8a-457e-bd84-688095221bdf None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:20.106090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1eca184c-da8a-457e-bd84-688095221bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:20.125059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca184c-da8a-457e-bd84-688095221bdf None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:20.129069 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd2ef0-16e6-4839-bb76-95b460670b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:20.129868 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd2ef0-16e6-4839-bb76-95b460670b5f service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:20.130156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd2ef0-16e6-4839-bb76-95b460670b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:20.130431 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd2ef0-16e6-4839-bb76-95b460670b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:20.139437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd2ef0-16e6-4839-bb76-95b460670b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:20.141041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd2ef0-16e6-4839-bb76-95b460670b5f service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:20.150643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1217/2435] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:20.333839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2172262-ed02-4c0e-ba88-c1cd8694b8a2 None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:20.334236 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2172262-ed02-4c0e-ba88-c1cd8694b8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:20.334570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2172262-ed02-4c0e-ba88-c1cd8694b8a2 None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:20.336229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1219/2436] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:32:20.341773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724640e-966a-414d-9915-6ef8c8be9254 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:20.342057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724640e-966a-414d-9915-6ef8c8be9254 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:20.342330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724640e-966a-414d-9915-6ef8c8be9254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:20.379045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d724640e-966a-414d-9915-6ef8c8be9254 None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:20.382782 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d724640e-966a-414d-9915-6ef8c8be9254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:20.400574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1218/2437] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:20.419408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2172262-ed02-4c0e-ba88-c1cd8694b8a2 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:20.424411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f2172262-ed02-4c0e-ba88-c1cd8694b8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:20.445222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2172262-ed02-4c0e-ba88-c1cd8694b8a2 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:20.450454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560c326-5913-410f-a867-2a5c51a8bcde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:20.451271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560c326-5913-410f-a867-2a5c51a8bcde service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:20.451759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560c326-5913-410f-a867-2a5c51a8bcde service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:20.452258 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560c326-5913-410f-a867-2a5c51a8bcde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:20.458799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcd58a-59d1-4917-8dbc-e6b8cbad81a4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:20.459647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcd58a-59d1-4917-8dbc-e6b8cbad81a4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:20.460363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcd58a-59d1-4917-8dbc-e6b8cbad81a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:20.467570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560c326-5913-410f-a867-2a5c51a8bcde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:20.470579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560c326-5913-410f-a867-2a5c51a8bcde service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:20.483955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-73fcd58a-59d1-4917-8dbc-e6b8cbad81a4 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:20.485743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-73fcd58a-59d1-4917-8dbc-e6b8cbad81a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:20.486153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1220/2438] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:20.557203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1219/2439] 10.222.0.62 () {58 vars in 967 bytes} [Mon Sep 5 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:32:20.570929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560c326-5913-410f-a867-2a5c51a8bcde service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:20.572434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c560c326-5913-410f-a867-2a5c51a8bcde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:20.658013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560c326-5913-410f-a867-2a5c51a8bcde service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:20.665356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1291929-8b9a-4224-a7fc-4148fbc3539d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:20.666118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1291929-8b9a-4224-a7fc-4148fbc3539d service swift] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:20.666575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1291929-8b9a-4224-a7fc-4148fbc3539d service swift] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:20.667047 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1291929-8b9a-4224-a7fc-4148fbc3539d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:20.677763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1291929-8b9a-4224-a7fc-4148fbc3539d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:20.680214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1291929-8b9a-4224-a7fc-4148fbc3539d service swift] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:20.708348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:20.708897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1221/2440] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:32:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:32:24.497115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcd58a-59d1-4917-8dbc-e6b8cbad81a4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:24.499273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-73fcd58a-59d1-4917-8dbc-e6b8cbad81a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:24.526912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1291929-8b9a-4224-a7fc-4148fbc3539d service swift] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:24.529504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a1291929-8b9a-4224-a7fc-4148fbc3539d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:24.545037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:24.596989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcd58a-59d1-4917-8dbc-e6b8cbad81a4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:24.603266 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:24.604115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:24.604474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:24.604821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] PATH_INFO: `/v3/users/a669f6cbac5a4d84a9e71c27c01969ff` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:24.606586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a669f6cbac5a4d84a9e71c27c01969ff)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:24.608655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:24.629044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1291929-8b9a-4224-a7fc-4148fbc3539d service swift] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:24.640029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee3978-2a48-40c5-8580-aea0698976ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:24.640749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee3978-2a48-40c5-8580-aea0698976ff service placement] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:24.641025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee3978-2a48-40c5-8580-aea0698976ff service placement] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:24.641301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee3978-2a48-40c5-8580-aea0698976ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:24.643211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a3ee3978-2a48-40c5-8580-aea0698976ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:24.656054 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee3978-2a48-40c5-8580-aea0698976ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:24.658505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee3978-2a48-40c5-8580-aea0698976ff service placement] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:24.673930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1222/2441] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:32:24.791997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a669f6cbac5a4d84a9e71c27c01969ff'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:24.792824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a669f6cbac5a4d84a9e71c27c01969ff'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:24.867726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a669f6cbac5a4d84a9e71c27c01969ff'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:24.883620 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a669f6cbac5a4d84a9e71c27c01969ff'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:24.884377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ca76b43b-7fd9-4b98-a63c-2185cb6e2e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a669f6cbac5a4d84a9e71c27c01969ff'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:24.898457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1220/2442] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:32:24 2022] DELETE /identity/v3/users/a669f6cbac5a4d84a9e71c27c01969ff => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:25.181349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee3978-2a48-40c5-8580-aea0698976ff service placement] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:25.199376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee3978-2a48-40c5-8580-aea0698976ff service placement] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:25.266869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:25.267504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:25.267815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:25.268974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] PATH_INFO: `/v3/projects/d5ecc5cd1d854b17bbe9ee678ecacfe3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:25.269341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ecc5cd1d854b17bbe9ee678ecacfe3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:25.271285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:25.365379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ecc5cd1d854b17bbe9ee678ecacfe3', 'request_id': 'req-08b92393-0196-493c-b85e-f82f954d3eb0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:25.371443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ecc5cd1d854b17bbe9ee678ecacfe3', 'request_id': 'req-08b92393-0196-493c-b85e-f82f954d3eb0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:25.521239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ecc5cd1d854b17bbe9ee678ecacfe3', 'request_id': 'req-08b92393-0196-493c-b85e-f82f954d3eb0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:25.545416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] The token cache is being invalidate because project d5ecc5cd1d854b17bbe9ee678ecacfe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:32:25.545416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:25.545416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08b92393-0196-493c-b85e-f82f954d3eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:25.545416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1223/2443] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:32:25 2022] DELETE /identity/v3/projects/d5ecc5cd1d854b17bbe9ee678ecacfe3 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:25.550216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42c42a-dc13-4e5e-859a-01924cc28b36 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:25.550981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42c42a-dc13-4e5e-859a-01924cc28b36 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:25.551398 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42c42a-dc13-4e5e-859a-01924cc28b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:25.577842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ab42c42a-dc13-4e5e-859a-01924cc28b36 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:25.579375 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ab42c42a-dc13-4e5e-859a-01924cc28b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:25.646634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1221/2444] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:32:25.655228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf77a79-63a0-4a31-84ec-19c8a74da1ff None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:25.655668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf77a79-63a0-4a31-84ec-19c8a74da1ff None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:25.656014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf77a79-63a0-4a31-84ec-19c8a74da1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:25.678281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-4cf77a79-63a0-4a31-84ec-19c8a74da1ff None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:25.679027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4cf77a79-63a0-4a31-84ec-19c8a74da1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:25.687807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1224/2445] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:25.692457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42c42a-dc13-4e5e-859a-01924cc28b36 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:25.703837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42c42a-dc13-4e5e-859a-01924cc28b36 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:25.708706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95ee81-89bd-4e2e-8fe4-f7221f41d126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:25.709461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ee81-89bd-4e2e-8fe4-f7221f41d126 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:25.709802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ee81-89bd-4e2e-8fe4-f7221f41d126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:25.710142 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ee81-89bd-4e2e-8fe4-f7221f41d126 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:25.710877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95ee81-89bd-4e2e-8fe4-f7221f41d126 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:32:25.711345 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95ee81-89bd-4e2e-8fe4-f7221f41d126 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:25.713191 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95ee81-89bd-4e2e-8fe4-f7221f41d126 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:25.729706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1222/2446] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:32:25.735013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf77a79-63a0-4a31-84ec-19c8a74da1ff None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:25.746245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf77a79-63a0-4a31-84ec-19c8a74da1ff None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:25.751558 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f24913-ab35-4260-8aec-f62979d60b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:25.752187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24913-ab35-4260-8aec-f62979d60b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:25.752615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24913-ab35-4260-8aec-f62979d60b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:25.752894 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24913-ab35-4260-8aec-f62979d60b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:25.753533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f24913-ab35-4260-8aec-f62979d60b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:25.755464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f24913-ab35-4260-8aec-f62979d60b9a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:25.770104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:25.904188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1225/2447] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:32:25 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:32:25.905474 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95ee81-89bd-4e2e-8fe4-f7221f41d126 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:25.918014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95ee81-89bd-4e2e-8fe4-f7221f41d126 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:25.949874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:25.981062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d4146-8567-4ecf-b5c5-99a76fbdc142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:25.981678 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4146-8567-4ecf-b5c5-99a76fbdc142 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:25.981956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4146-8567-4ecf-b5c5-99a76fbdc142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:25.982247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4146-8567-4ecf-b5c5-99a76fbdc142 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:25.982883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d4146-8567-4ecf-b5c5-99a76fbdc142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:25.984889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d4146-8567-4ecf-b5c5-99a76fbdc142 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:26.075646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1223/2448] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:32:25 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:32:26.079967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f24913-ab35-4260-8aec-f62979d60b9a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:26.090873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f24913-ab35-4260-8aec-f62979d60b9a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:26.097583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea7c9f-0771-43e2-a01f-e0c24356e9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:26.098381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea7c9f-0771-43e2-a01f-e0c24356e9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:26.098907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea7c9f-0771-43e2-a01f-e0c24356e9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:26.099180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea7c9f-0771-43e2-a01f-e0c24356e9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:26.100402 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea7c9f-0771-43e2-a01f-e0c24356e9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:26.102013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea7c9f-0771-43e2-a01f-e0c24356e9d5 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:26.113606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1226/2449] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:32:26.118429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d4146-8567-4ecf-b5c5-99a76fbdc142 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:26.130267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d4146-8567-4ecf-b5c5-99a76fbdc142 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:26.135009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a939a91-abba-47ea-a2cd-af1a48cdc6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:26.136853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a939a91-abba-47ea-a2cd-af1a48cdc6db admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:26.136853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a939a91-abba-47ea-a2cd-af1a48cdc6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:26.136853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a939a91-abba-47ea-a2cd-af1a48cdc6db admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:26.137108 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a939a91-abba-47ea-a2cd-af1a48cdc6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:26.138517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a939a91-abba-47ea-a2cd-af1a48cdc6db admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:26.150076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1224/2450] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:32:26.154478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea7c9f-0771-43e2-a01f-e0c24356e9d5 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:26.165953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea7c9f-0771-43e2-a01f-e0c24356e9d5 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:26.172406 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530df40-d377-4695-bda2-58db0d902a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:26.173089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530df40-d377-4695-bda2-58db0d902a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:26.173368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530df40-d377-4695-bda2-58db0d902a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:26.173786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530df40-d377-4695-bda2-58db0d902a46 admin admin] PATH_INFO: `/v3/projects/945b7831a2874882b10c8561f0e37164/users/54a7212d11a24e2ab3e4ba371d2123b5/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:26.190007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530df40-d377-4695-bda2-58db0d902a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=945b7831a2874882b10c8561f0e37164, user_id=54a7212d11a24e2ab3e4ba371d2123b5, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:26.192735 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530df40-d377-4695-bda2-58db0d902a46 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:26.301734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1227/2451] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:32:26 2022] PUT /identity/v3/projects/945b7831a2874882b10c8561f0e37164/users/54a7212d11a24e2ab3e4ba371d2123b5/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:32:26.304930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffef081-276b-4f6d-9d7a-8dde5a0ce24c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:26.305213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffef081-276b-4f6d-9d7a-8dde5a0ce24c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:26.305484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffef081-276b-4f6d-9d7a-8dde5a0ce24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:26.341232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9ffef081-276b-4f6d-9d7a-8dde5a0ce24c None None] MFA Rules not processed for user `54a7212d11a24e2ab3e4ba371d2123b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:26.343234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9ffef081-276b-4f6d-9d7a-8dde5a0ce24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:26.447313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1225/2452] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:32:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:26.458918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530df40-d377-4695-bda2-58db0d902a46 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:26.461244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7530df40-d377-4695-bda2-58db0d902a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:26.530184 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530df40-d377-4695-bda2-58db0d902a46 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:26.535197 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec33b9-5cfa-464d-ad3f-95925ddf6c2d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '237bbeebe2a949aba444353c1d2ee04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:26.536081 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec33b9-5cfa-464d-ad3f-95925ddf6c2d service glance] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:26.536461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec33b9-5cfa-464d-ad3f-95925ddf6c2d service glance] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:26.536874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec33b9-5cfa-464d-ad3f-95925ddf6c2d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:26.547198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec33b9-5cfa-464d-ad3f-95925ddf6c2d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:26.548707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec33b9-5cfa-464d-ad3f-95925ddf6c2d service glance] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:26.559604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1228/2453] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:32:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:32:26.987686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f003323-9912-42e8-be99-fcb62d786934 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:26.988267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f003323-9912-42e8-be99-fcb62d786934 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:26.988267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f003323-9912-42e8-be99-fcb62d786934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:27.011290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-9f003323-9912-42e8-be99-fcb62d786934 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:27.012562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9f003323-9912-42e8-be99-fcb62d786934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:27.083091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1226/2454] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:32:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:32:27.095334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec33b9-5cfa-464d-ad3f-95925ddf6c2d service glance] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:27.096780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e3ec33b9-5cfa-464d-ad3f-95925ddf6c2d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:27.169923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec33b9-5cfa-464d-ad3f-95925ddf6c2d service glance] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:27.179789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33b487-8ef5-4ca4-bb7d-f96eaf736b9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:27.180538 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33b487-8ef5-4ca4-bb7d-f96eaf736b9b service swift] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:27.180744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33b487-8ef5-4ca4-bb7d-f96eaf736b9b service swift] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:27.181025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33b487-8ef5-4ca4-bb7d-f96eaf736b9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:27.191646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33b487-8ef5-4ca4-bb7d-f96eaf736b9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:27.193560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33b487-8ef5-4ca4-bb7d-f96eaf736b9b service swift] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:27.205203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1229/2455] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:32:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:32:37.076100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:37.200960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=76,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:32:52.110485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2e4ab-01b8-4c91-9538-700aad8b8ef6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:52.110485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2e4ab-01b8-4c91-9538-700aad8b8ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:52.110485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2e4ab-01b8-4c91-9538-700aad8b8ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:52.130994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e6f2e4ab-01b8-4c91-9538-700aad8b8ef6 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:52.132325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e6f2e4ab-01b8-4c91-9538-700aad8b8ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:52.141001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1227/2456] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:52.162639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33b487-8ef5-4ca4-bb7d-f96eaf736b9b service swift] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:52.173851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33b487-8ef5-4ca4-bb7d-f96eaf736b9b service swift] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:52.179004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b25fd6-5861-47cc-8d6d-b1360679a6a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:52.179643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b25fd6-5861-47cc-8d6d-b1360679a6a3 service swift] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:52.179990 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b25fd6-5861-47cc-8d6d-b1360679a6a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:52.180290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b25fd6-5861-47cc-8d6d-b1360679a6a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:52.189757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b25fd6-5861-47cc-8d6d-b1360679a6a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:52.191425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b25fd6-5861-47cc-8d6d-b1360679a6a3 service swift] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:52.200145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1230/2457] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:32:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:32:53.086802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d737fb7-2e44-4184-b6f5-dca3089b38d7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:53.087076 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d737fb7-2e44-4184-b6f5-dca3089b38d7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:53.087348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d737fb7-2e44-4184-b6f5-dca3089b38d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:53.108502 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-0d737fb7-2e44-4184-b6f5-dca3089b38d7 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:53.109725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0d737fb7-2e44-4184-b6f5-dca3089b38d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:53.117055 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1228/2458] 10.222.0.62 () {58 vars in 967 bytes} [Mon Sep 5 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:53.130007 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b25fd6-5861-47cc-8d6d-b1360679a6a3 service swift] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:53.142077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b25fd6-5861-47cc-8d6d-b1360679a6a3 service swift] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:53.147602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a984e84-67f7-418e-89e2-29a59e2d82c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:53.148241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a984e84-67f7-418e-89e2-29a59e2d82c4 service swift] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:53.148520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a984e84-67f7-418e-89e2-29a59e2d82c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:53.148795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a984e84-67f7-418e-89e2-29a59e2d82c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:53.157789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a984e84-67f7-418e-89e2-29a59e2d82c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:53.160230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a984e84-67f7-418e-89e2-29a59e2d82c4 service swift] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:53.169043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1231/2459] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:32:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:32:53.880683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4199c9-6370-453c-8fff-37fbd9316971 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:53.880959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4199c9-6370-453c-8fff-37fbd9316971 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:53.881232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4199c9-6370-453c-8fff-37fbd9316971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:53.901322 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-8d4199c9-6370-453c-8fff-37fbd9316971 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:53.902566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-8d4199c9-6370-453c-8fff-37fbd9316971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:53.909879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1229/2460] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:32:53.922523 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a984e84-67f7-418e-89e2-29a59e2d82c4 service swift] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:53.933240 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a984e84-67f7-418e-89e2-29a59e2d82c4 service swift] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:53.938471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a016d-a648-41f6-ad5c-ade2711b32f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:53.939074 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a016d-a648-41f6-ad5c-ade2711b32f7 service swift] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:53.939350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a016d-a648-41f6-ad5c-ade2711b32f7 service swift] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:53.939659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a016d-a648-41f6-ad5c-ade2711b32f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:53.947978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a016d-a648-41f6-ad5c-ade2711b32f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:53.949563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a016d-a648-41f6-ad5c-ade2711b32f7 service swift] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:53.957840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1232/2461] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:32:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:32:54.617051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ab83-28a7-43e7-b3d9-2610a782733d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:54.617520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ab83-28a7-43e7-b3d9-2610a782733d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:54.617614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ab83-28a7-43e7-b3d9-2610a782733d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:54.638088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2a38ab83-28a7-43e7-b3d9-2610a782733d None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:54.639422 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2a38ab83-28a7-43e7-b3d9-2610a782733d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:54.646718 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1230/2462] 10.222.0.62 () {58 vars in 967 bytes} [Mon Sep 5 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:54.659195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a016d-a648-41f6-ad5c-ade2711b32f7 service swift] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:54.669803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a016d-a648-41f6-ad5c-ade2711b32f7 service swift] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:54.675457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7b12b-20f2-470d-bbe3-273e884bff8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:54.678792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7b12b-20f2-470d-bbe3-273e884bff8f service swift] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:54.678792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7b12b-20f2-470d-bbe3-273e884bff8f service swift] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:54.678792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7b12b-20f2-470d-bbe3-273e884bff8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:54.685676 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7b12b-20f2-470d-bbe3-273e884bff8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:54.687255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7b12b-20f2-470d-bbe3-273e884bff8f service swift] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:54.695149 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1233/2463] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:32:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:32:55.756890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350228ff-d9c6-4817-ab12-f1d84b1c4fe8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:55.757394 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350228ff-d9c6-4817-ab12-f1d84b1c4fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:55.757498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350228ff-d9c6-4817-ab12-f1d84b1c4fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:55.779273 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-350228ff-d9c6-4817-ab12-f1d84b1c4fe8 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:32:55.780631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-350228ff-d9c6-4817-ab12-f1d84b1c4fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:55.788262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1231/2464] 10.222.0.62 () {58 vars in 967 bytes} [Mon Sep 5 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:32:55.800497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7b12b-20f2-470d-bbe3-273e884bff8f service swift] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:55.812390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7b12b-20f2-470d-bbe3-273e884bff8f service swift] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:55.817195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c4e5f-1ad5-4373-90d7-694881b811b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:55.817804 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c4e5f-1ad5-4373-90d7-694881b811b3 service swift] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:55.818084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c4e5f-1ad5-4373-90d7-694881b811b3 service swift] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:55.818413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c4e5f-1ad5-4373-90d7-694881b811b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:55.828389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c4e5f-1ad5-4373-90d7-694881b811b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:55.829993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c4e5f-1ad5-4373-90d7-694881b811b3 service swift] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:55.839668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1234/2465] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:32:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:32:59.690466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350228ff-d9c6-4817-ab12-f1d84b1c4fe8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:32:59.691974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-350228ff-d9c6-4817-ab12-f1d84b1c4fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:32:59.757163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350228ff-d9c6-4817-ab12-f1d84b1c4fe8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:32:59.761736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:32:59.762363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:32:59.762645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:32:59.762922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] PATH_INFO: `/v3/users/d86ccae798764807976ab6c52b5f50a7` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:32:59.764096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86ccae798764807976ab6c52b5f50a7)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:32:59.766016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:32:59.933309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86ccae798764807976ab6c52b5f50a7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:32:59.933972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86ccae798764807976ab6c52b5f50a7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:00.028327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86ccae798764807976ab6c52b5f50a7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:00.041553 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86ccae798764807976ab6c52b5f50a7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:00.042215 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86ccae798764807976ab6c52b5f50a7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:00.054890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1232/2466] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:32:59 2022] DELETE /identity/v3/users/d86ccae798764807976ab6c52b5f50a7 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:33:00.059349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4f578-a863-4fa9-a58e-cc1fba3b9e4a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:00.059818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4f578-a863-4fa9-a58e-cc1fba3b9e4a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:00.059937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4f578-a863-4fa9-a58e-cc1fba3b9e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:00.080241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-04e4f578-a863-4fa9-a58e-cc1fba3b9e4a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:00.081449 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-04e4f578-a863-4fa9-a58e-cc1fba3b9e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:00.140839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1235/2467] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:00.151086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:00.152739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:00.208510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:00.214956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aefabf-696d-4a8d-a0ee-80cffd6c98cf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:00.222033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3fc9a-ec3d-4c64-af89-c0da0553a6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:00.222033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3fc9a-ec3d-4c64-af89-c0da0553a6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:00.222033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3fc9a-ec3d-4c64-af89-c0da0553a6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:00.222033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3fc9a-ec3d-4c64-af89-c0da0553a6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:00.234806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3fc9a-ec3d-4c64-af89-c0da0553a6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:00.236265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3fc9a-ec3d-4c64-af89-c0da0553a6c8 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:00.247591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1233/2468] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:33:00.494371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4f578-a863-4fa9-a58e-cc1fba3b9e4a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:00.499788 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-04e4f578-a863-4fa9-a58e-cc1fba3b9e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:00.512285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4f578-a863-4fa9-a58e-cc1fba3b9e4a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:00.519305 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:00.520180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:00.520180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:00.520650 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] PATH_INFO: `/v3/projects/29ac989524a34d65bcefc990244d02dd` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:00.521690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ac989524a34d65bcefc990244d02dd)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:00.523709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:00.652412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ac989524a34d65bcefc990244d02dd', 'request_id': 'req-e663212a-535f-48be-b671-9800b94c2d3e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:00.658004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ac989524a34d65bcefc990244d02dd', 'request_id': 'req-e663212a-535f-48be-b671-9800b94c2d3e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:00.725712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ac989524a34d65bcefc990244d02dd', 'request_id': 'req-e663212a-535f-48be-b671-9800b94c2d3e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:00.738228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] The token cache is being invalidate because project 29ac989524a34d65bcefc990244d02dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:33:00.738451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:00.739084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e663212a-535f-48be-b671-9800b94c2d3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:00.740602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1236/2469] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:33:00 2022] DELETE /identity/v3/projects/29ac989524a34d65bcefc990244d02dd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:33:05.426901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3fc9a-ec3d-4c64-af89-c0da0553a6c8 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:05.428336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1de3fc9a-ec3d-4c64-af89-c0da0553a6c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:05.441490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3fc9a-ec3d-4c64-af89-c0da0553a6c8 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:05.498199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:05.498823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:05.499096 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:05.499365 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] PATH_INFO: `/v3/users/54a7212d11a24e2ab3e4ba371d2123b5` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:05.500603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a7212d11a24e2ab3e4ba371d2123b5)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:05.502572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:05.699080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a7212d11a24e2ab3e4ba371d2123b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:05.699706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a7212d11a24e2ab3e4ba371d2123b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:05.765054 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a7212d11a24e2ab3e4ba371d2123b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:05.778108 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a7212d11a24e2ab3e4ba371d2123b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:05.778743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a7212d11a24e2ab3e4ba371d2123b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:05.791366 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1234/2470] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:33:05 2022] DELETE /identity/v3/users/54a7212d11a24e2ab3e4ba371d2123b5 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:33:05.796032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa69dfd-6614-42c4-92fa-bbb22f6e8e13 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:05.796547 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa69dfd-6614-42c4-92fa-bbb22f6e8e13 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:05.796652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa69dfd-6614-42c4-92fa-bbb22f6e8e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:05.816021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1aa69dfd-6614-42c4-92fa-bbb22f6e8e13 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:05.817232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1aa69dfd-6614-42c4-92fa-bbb22f6e8e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:05.828919 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:05.875604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1237/2471] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:05.885320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:05.889189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:05.944057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadfb46-3678-4aeb-b607-692e822a6ebe admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:05.947922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab17287-6be4-4210-a55b-29891c0b605a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:05.948605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab17287-6be4-4210-a55b-29891c0b605a service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:05.948880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab17287-6be4-4210-a55b-29891c0b605a service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:05.949202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab17287-6be4-4210-a55b-29891c0b605a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:05.958255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab17287-6be4-4210-a55b-29891c0b605a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:05.959696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab17287-6be4-4210-a55b-29891c0b605a service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:05.969603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1235/2472] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:33:06.362564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa69dfd-6614-42c4-92fa-bbb22f6e8e13 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:06.364190 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1aa69dfd-6614-42c4-92fa-bbb22f6e8e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:06.376503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa69dfd-6614-42c4-92fa-bbb22f6e8e13 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:06.381162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:06.381832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:06.382113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:06.382390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] PATH_INFO: `/v3/projects/945b7831a2874882b10c8561f0e37164` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:06.383555 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=945b7831a2874882b10c8561f0e37164)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:06.386038 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:06.530464 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '945b7831a2874882b10c8561f0e37164', 'request_id': 'req-a71482c9-8639-4548-9bdc-4df26ae2f8b0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:06.536951 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '945b7831a2874882b10c8561f0e37164', 'request_id': 'req-a71482c9-8639-4548-9bdc-4df26ae2f8b0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:06.616254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '945b7831a2874882b10c8561f0e37164', 'request_id': 'req-a71482c9-8639-4548-9bdc-4df26ae2f8b0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:06.628958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] The token cache is being invalidate because project 945b7831a2874882b10c8561f0e37164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:33:06.629249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:06.629924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-a71482c9-8639-4548-9bdc-4df26ae2f8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:06.631431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1238/2473] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:33:06 2022] DELETE /identity/v3/projects/945b7831a2874882b10c8561f0e37164 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:33:06.642733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89173bdc-1128-44dc-962f-94287e280d6d None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:06.643313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89173bdc-1128-44dc-962f-94287e280d6d None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:06.643313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89173bdc-1128-44dc-962f-94287e280d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:06.663265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-89173bdc-1128-44dc-962f-94287e280d6d None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:06.664492 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-89173bdc-1128-44dc-962f-94287e280d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:06.669904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:06.724353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1236/2474] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:06.733703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd690e4-fa8b-4f71-9b61-7cf34da0a6b8 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:06.737559 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd690e4-fa8b-4f71-9b61-7cf34da0a6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:06.737559 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd690e4-fa8b-4f71-9b61-7cf34da0a6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:06.754725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ddd690e4-fa8b-4f71-9b61-7cf34da0a6b8 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:06.755988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ddd690e4-fa8b-4f71-9b61-7cf34da0a6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:06.764573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1239/2475] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:06.768877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89173bdc-1128-44dc-962f-94287e280d6d None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:06.778840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89173bdc-1128-44dc-962f-94287e280d6d None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:06.783832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3106fd-5a95-4cbb-89d4-ca6fcd0abcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:06.784395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3106fd-5a95-4cbb-89d4-ca6fcd0abcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:06.785050 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3106fd-5a95-4cbb-89d4-ca6fcd0abcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:06.785326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3106fd-5a95-4cbb-89d4-ca6fcd0abcb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:06.785980 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3106fd-5a95-4cbb-89d4-ca6fcd0abcb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:33:06.786284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3106fd-5a95-4cbb-89d4-ca6fcd0abcb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:06.787921 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3106fd-5a95-4cbb-89d4-ca6fcd0abcb7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:06.799627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1237/2476] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:33:06.803925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd690e4-fa8b-4f71-9b61-7cf34da0a6b8 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:06.814112 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd690e4-fa8b-4f71-9b61-7cf34da0a6b8 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:06.818944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01bd7f-cd7c-44a4-92cc-2909b27cadd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:06.823563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01bd7f-cd7c-44a4-92cc-2909b27cadd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:06.823563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01bd7f-cd7c-44a4-92cc-2909b27cadd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:06.823563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01bd7f-cd7c-44a4-92cc-2909b27cadd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:06.823563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01bd7f-cd7c-44a4-92cc-2909b27cadd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:06.823563 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01bd7f-cd7c-44a4-92cc-2909b27cadd2 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:06.939362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1240/2477] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:33:06 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:33:06.943564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3106fd-5a95-4cbb-89d4-ca6fcd0abcb7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:06.954260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3106fd-5a95-4cbb-89d4-ca6fcd0abcb7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:07.011737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c657641-184f-4ce5-86f8-79293adf78a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:07.012336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c657641-184f-4ce5-86f8-79293adf78a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:07.012610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c657641-184f-4ce5-86f8-79293adf78a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:07.012881 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c657641-184f-4ce5-86f8-79293adf78a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:07.013494 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c657641-184f-4ce5-86f8-79293adf78a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:07.015430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c657641-184f-4ce5-86f8-79293adf78a4 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:07.115220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1238/2478] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:33:06 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:33:07.120027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01bd7f-cd7c-44a4-92cc-2909b27cadd2 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:07.130662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01bd7f-cd7c-44a4-92cc-2909b27cadd2 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:07.135356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40579e6f-2042-46ee-9245-4ffedd3d4959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:07.135994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40579e6f-2042-46ee-9245-4ffedd3d4959 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:07.136273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40579e6f-2042-46ee-9245-4ffedd3d4959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:07.136547 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40579e6f-2042-46ee-9245-4ffedd3d4959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:07.137182 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40579e6f-2042-46ee-9245-4ffedd3d4959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:07.138777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40579e6f-2042-46ee-9245-4ffedd3d4959 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:07.148633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1241/2479] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:33:07.152542 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c657641-184f-4ce5-86f8-79293adf78a4 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:07.162566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c657641-184f-4ce5-86f8-79293adf78a4 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:07.167866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a7b26-f2bf-47ad-a965-0c633a0a9967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:07.168465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a7b26-f2bf-47ad-a965-0c633a0a9967 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:07.168746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a7b26-f2bf-47ad-a965-0c633a0a9967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:07.169018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a7b26-f2bf-47ad-a965-0c633a0a9967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:07.169651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a7b26-f2bf-47ad-a965-0c633a0a9967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:07.171244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a7b26-f2bf-47ad-a965-0c633a0a9967 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:07.182340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1239/2480] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:33:07.186679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40579e6f-2042-46ee-9245-4ffedd3d4959 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:07.196411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40579e6f-2042-46ee-9245-4ffedd3d4959 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:07.201105 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4557b-7d66-451d-aff2-cd67506dc55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:07.201742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4557b-7d66-451d-aff2-cd67506dc55a admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:07.202023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4557b-7d66-451d-aff2-cd67506dc55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:07.202298 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4557b-7d66-451d-aff2-cd67506dc55a admin admin] PATH_INFO: `/v3/projects/6123e8ffd3174e62aeed92fbf73b7193/users/46de766909c14cc48113cb1a3ae03b17/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:07.216477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4557b-7d66-451d-aff2-cd67506dc55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6123e8ffd3174e62aeed92fbf73b7193, user_id=46de766909c14cc48113cb1a3ae03b17, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:07.219134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4557b-7d66-451d-aff2-cd67506dc55a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:07.293462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1242/2481] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:33:07 2022] PUT /identity/v3/projects/6123e8ffd3174e62aeed92fbf73b7193/users/46de766909c14cc48113cb1a3ae03b17/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:33:07.304445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec4e13-fc52-4976-b8d6-c15aec7dfe87 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:07.304665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec4e13-fc52-4976-b8d6-c15aec7dfe87 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:07.304931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec4e13-fc52-4976-b8d6-c15aec7dfe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:07.333598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-99ec4e13-fc52-4976-b8d6-c15aec7dfe87 None None] MFA Rules not processed for user `46de766909c14cc48113cb1a3ae03b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:07.334810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-99ec4e13-fc52-4976-b8d6-c15aec7dfe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:07.398748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1240/2482] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:07.410508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4557b-7d66-451d-aff2-cd67506dc55a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:07.412227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5be4557b-7d66-451d-aff2-cd67506dc55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:07.453006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:07.467827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4557b-7d66-451d-aff2-cd67506dc55a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:07.471452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ed64e-1aa0-43cb-ac3e-afb12718718f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:07.472080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ed64e-1aa0-43cb-ac3e-afb12718718f service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:07.472373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ed64e-1aa0-43cb-ac3e-afb12718718f service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:07.472630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ed64e-1aa0-43cb-ac3e-afb12718718f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:07.481695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ed64e-1aa0-43cb-ac3e-afb12718718f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:07.483382 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ed64e-1aa0-43cb-ac3e-afb12718718f service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:07.492075 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1243/2483] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:33:17.395870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:17.514470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=9 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:31.215571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bc046-dad9-436e-a216-3f19e7769048 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:31.216992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bc046-dad9-436e-a216-3f19e7769048 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:31.216992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bc046-dad9-436e-a216-3f19e7769048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:31.237226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7a9bc046-dad9-436e-a216-3f19e7769048 None None] MFA Rules not processed for user `de2273220e824339bd13d968c75df26b`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:31.239208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7a9bc046-dad9-436e-a216-3f19e7769048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:31.296487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1241/2484] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:33:41.289202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=25 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:44.515675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce82233-8f93-4bc4-8140-eb1ffbf56865 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:44.516250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce82233-8f93-4bc4-8140-eb1ffbf56865 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:44.516250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce82233-8f93-4bc4-8140-eb1ffbf56865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:44.536699 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1ce82233-8f93-4bc4-8140-eb1ffbf56865 None None] MFA Rules not processed for user `f5543fbd902d441cb6f03d3155f1af8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:44.537941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1ce82233-8f93-4bc4-8140-eb1ffbf56865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:44.597220 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1244/2485] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:47.618959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bc046-dad9-436e-a216-3f19e7769048 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:47.693230 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bc046-dad9-436e-a216-3f19e7769048 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:47.698218 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:47.698930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:47.699277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:47.700209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] PATH_INFO: `/v3/users/46de766909c14cc48113cb1a3ae03b17` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:47.701573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46de766909c14cc48113cb1a3ae03b17)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:47.704000 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:47.931397 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46de766909c14cc48113cb1a3ae03b17'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:47.932119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46de766909c14cc48113cb1a3ae03b17'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:48.001348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46de766909c14cc48113cb1a3ae03b17'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:48.015992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46de766909c14cc48113cb1a3ae03b17'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:48.016777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46de766909c14cc48113cb1a3ae03b17'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:48.030367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1242/2486] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:33:47 2022] DELETE /identity/v3/users/46de766909c14cc48113cb1a3ae03b17 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:33:48.035283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416eb5-46d4-4146-b75d-ae7e04d392ca None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:48.035910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416eb5-46d4-4146-b75d-ae7e04d392ca None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:48.035910 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416eb5-46d4-4146-b75d-ae7e04d392ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:48.060077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-90416eb5-46d4-4146-b75d-ae7e04d392ca None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:48.061383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-90416eb5-46d4-4146-b75d-ae7e04d392ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:48.131368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1245/2487] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:48.141248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:48.142872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:48.203705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100d905-fe8a-4de9-937a-032ce2e457e2 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:48.207922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edd194-f860-4c60-9342-570214dcd0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:48.208687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd194-f860-4c60-9342-570214dcd0aa service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:48.209062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd194-f860-4c60-9342-570214dcd0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:48.209452 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd194-f860-4c60-9342-570214dcd0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:48.221716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edd194-f860-4c60-9342-570214dcd0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:48.223299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edd194-f860-4c60-9342-570214dcd0aa service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:48.234064 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1243/2488] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:33:48.724648 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90416eb5-46d4-4146-b75d-ae7e04d392ca None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:48.727066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-90416eb5-46d4-4146-b75d-ae7e04d392ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:48.752226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90416eb5-46d4-4146-b75d-ae7e04d392ca None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:48.759190 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:48.760836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:48.770611 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:48.771050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] PATH_INFO: `/v3/projects/6123e8ffd3174e62aeed92fbf73b7193` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:48.773572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6123e8ffd3174e62aeed92fbf73b7193)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:48.776397 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:48.911270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6123e8ffd3174e62aeed92fbf73b7193', 'request_id': 'req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:48.916989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6123e8ffd3174e62aeed92fbf73b7193', 'request_id': 'req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:48.984596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6123e8ffd3174e62aeed92fbf73b7193', 'request_id': 'req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:48.997986 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] The token cache is being invalidate because project 6123e8ffd3174e62aeed92fbf73b7193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:33:48.998277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:48.999523 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5c0e6e34-2688-4a17-8ee3-5b0dccd35195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:33:49.001100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1246/2489] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:33:48 2022] DELETE /identity/v3/projects/6123e8ffd3174e62aeed92fbf73b7193 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:33:49.006326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d8f1-c969-46f2-8bf6-031cdce932c8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:49.006880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d8f1-c969-46f2-8bf6-031cdce932c8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:49.006880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d8f1-c969-46f2-8bf6-031cdce932c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:49.028608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d649d8f1-c969-46f2-8bf6-031cdce932c8 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:49.029845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d649d8f1-c969-46f2-8bf6-031cdce932c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:49.055939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:49.111069 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1244/2490] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:49.113388 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a124d7-edb5-4a6d-b090-c834f415b513 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:49.114203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a124d7-edb5-4a6d-b090-c834f415b513 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:49.114480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a124d7-edb5-4a6d-b090-c834f415b513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:49.135795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-56a124d7-edb5-4a6d-b090-c834f415b513 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:49.137066 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-56a124d7-edb5-4a6d-b090-c834f415b513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:49.146188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1247/2491] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:49.150551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d8f1-c969-46f2-8bf6-031cdce932c8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:49.161047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d8f1-c969-46f2-8bf6-031cdce932c8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:49.167720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f545ccc-f2a2-41e1-b7b3-9167733c23d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:49.168324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545ccc-f2a2-41e1-b7b3-9167733c23d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:49.168890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545ccc-f2a2-41e1-b7b3-9167733c23d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:49.169051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545ccc-f2a2-41e1-b7b3-9167733c23d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:49.170635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f545ccc-f2a2-41e1-b7b3-9167733c23d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:33:49.170944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f545ccc-f2a2-41e1-b7b3-9167733c23d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:49.172751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f545ccc-f2a2-41e1-b7b3-9167733c23d1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:49.190607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1245/2492] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:33:49.191642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a124d7-edb5-4a6d-b090-c834f415b513 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:49.201514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a124d7-edb5-4a6d-b090-c834f415b513 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:49.206405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447da1c-aabc-4a5e-9e27-8b9ff5dd8b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:49.207028 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447da1c-aabc-4a5e-9e27-8b9ff5dd8b8b admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:49.207308 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447da1c-aabc-4a5e-9e27-8b9ff5dd8b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:49.212716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447da1c-aabc-4a5e-9e27-8b9ff5dd8b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:49.212716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447da1c-aabc-4a5e-9e27-8b9ff5dd8b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:49.212716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447da1c-aabc-4a5e-9e27-8b9ff5dd8b8b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:49.311361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:49.318583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1248/2493] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:33:49 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:33:49.326544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f545ccc-f2a2-41e1-b7b3-9167733c23d1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:49.337166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f545ccc-f2a2-41e1-b7b3-9167733c23d1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:49.406932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb7a8c-d31c-442f-b117-7137738dacd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:49.407731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb7a8c-d31c-442f-b117-7137738dacd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:49.408011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb7a8c-d31c-442f-b117-7137738dacd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:49.408353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb7a8c-d31c-442f-b117-7137738dacd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:49.409292 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb7a8c-d31c-442f-b117-7137738dacd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:49.411524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb7a8c-d31c-442f-b117-7137738dacd9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:49.527771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1246/2494] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:33:49 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:33:49.533133 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447da1c-aabc-4a5e-9e27-8b9ff5dd8b8b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:49.544293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447da1c-aabc-4a5e-9e27-8b9ff5dd8b8b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:49.555395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c2b5f-38fd-484d-bdc8-87e944a4d0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:49.556058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2b5f-38fd-484d-bdc8-87e944a4d0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:49.556333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2b5f-38fd-484d-bdc8-87e944a4d0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:49.558492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2b5f-38fd-484d-bdc8-87e944a4d0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:49.558492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c2b5f-38fd-484d-bdc8-87e944a4d0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:49.558858 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c2b5f-38fd-484d-bdc8-87e944a4d0c8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:49.584985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1249/2495] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:33:49.586004 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb7a8c-d31c-442f-b117-7137738dacd9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:49.602402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb7a8c-d31c-442f-b117-7137738dacd9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:49.610478 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3dd7b-6786-4422-8146-fb6d2177b968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:49.611788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3dd7b-6786-4422-8146-fb6d2177b968 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:49.612305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3dd7b-6786-4422-8146-fb6d2177b968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:49.612799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3dd7b-6786-4422-8146-fb6d2177b968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:49.613930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3dd7b-6786-4422-8146-fb6d2177b968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:49.617252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3dd7b-6786-4422-8146-fb6d2177b968 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:49.638518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1247/2496] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:33:49.645051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c2b5f-38fd-484d-bdc8-87e944a4d0c8 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:49.659193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c2b5f-38fd-484d-bdc8-87e944a4d0c8 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:49.665721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1873629-f732-4b6b-90a1-90c7419be6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:49.666582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1873629-f732-4b6b-90a1-90c7419be6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:49.666964 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1873629-f732-4b6b-90a1-90c7419be6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:49.667346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1873629-f732-4b6b-90a1-90c7419be6cf admin admin] PATH_INFO: `/v3/projects/6b86bf1c78d74ceaaff6ead90a70526f/users/087ceea13e3747f49096b42d9d5c1c86/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:49.685380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1873629-f732-4b6b-90a1-90c7419be6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b86bf1c78d74ceaaff6ead90a70526f, user_id=087ceea13e3747f49096b42d9d5c1c86, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:49.688272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1873629-f732-4b6b-90a1-90c7419be6cf admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:49.783516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1250/2497] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:33:49 2022] PUT /identity/v3/projects/6b86bf1c78d74ceaaff6ead90a70526f/users/087ceea13e3747f49096b42d9d5c1c86/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:33:49.785698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd03a9-c4b0-415d-b590-0ea8f2366726 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:49.785978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd03a9-c4b0-415d-b590-0ea8f2366726 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:49.788507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd03a9-c4b0-415d-b590-0ea8f2366726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:49.821647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-3acd03a9-c4b0-415d-b590-0ea8f2366726 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:49.823786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3acd03a9-c4b0-415d-b590-0ea8f2366726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:49.919372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1248/2498] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:49.926143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1873629-f732-4b6b-90a1-90c7419be6cf admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:49.931762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e1873629-f732-4b6b-90a1-90c7419be6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:50.061014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1873629-f732-4b6b-90a1-90c7419be6cf admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:50.074594 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea2253-ea1a-4235-9b80-931f2e6396b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:50.075757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea2253-ea1a-4235-9b80-931f2e6396b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:50.076638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea2253-ea1a-4235-9b80-931f2e6396b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:50.077034 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea2253-ea1a-4235-9b80-931f2e6396b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:50.100138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea2253-ea1a-4235-9b80-931f2e6396b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:50.101724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea2253-ea1a-4235-9b80-931f2e6396b8 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:50.117462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1251/2499] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:33:56.229117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad8fec-b376-4a3d-8ae8-eb7831dccce7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:56.229733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad8fec-b376-4a3d-8ae8-eb7831dccce7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:56.230175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad8fec-b376-4a3d-8ae8-eb7831dccce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:56.253500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d0ad8fec-b376-4a3d-8ae8-eb7831dccce7 None None] MFA Rules not processed for user `9a61a18900c44bea89647fc9ccdaf110`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:56.254726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d0ad8fec-b376-4a3d-8ae8-eb7831dccce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:56.284002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:56.327587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1249/2500] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:33:57.565530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc8507-fb27-4f47-9ab9-1044d5c83a8e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:57.566341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc8507-fb27-4f47-9ab9-1044d5c83a8e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:57.566341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc8507-fb27-4f47-9ab9-1044d5c83a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:57.597207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ddcc8507-fb27-4f47-9ab9-1044d5c83a8e None None] MFA Rules not processed for user `087ceea13e3747f49096b42d9d5c1c86`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:33:57.598678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ddcc8507-fb27-4f47-9ab9-1044d5c83a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:33:57.724934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1252/2501] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:33:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:33:57.729699 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad8fec-b376-4a3d-8ae8-eb7831dccce7 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:57.746934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad8fec-b376-4a3d-8ae8-eb7831dccce7 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:57.754234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31e2ba-e2d2-4961-aecc-c8269d26c42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:57.755049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31e2ba-e2d2-4961-aecc-c8269d26c42f admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:57.755853 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31e2ba-e2d2-4961-aecc-c8269d26c42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:57.756139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31e2ba-e2d2-4961-aecc-c8269d26c42f admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:57.757750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31e2ba-e2d2-4961-aecc-c8269d26c42f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:57.762408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31e2ba-e2d2-4961-aecc-c8269d26c42f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:57.860601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1250/2502] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:33:57 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:33:57.864765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc8507-fb27-4f47-9ab9-1044d5c83a8e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:57.876151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc8507-fb27-4f47-9ab9-1044d5c83a8e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:57.930064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:33:57.938245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2e0d3-8575-44fb-b373-035534904600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:57.938872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2e0d3-8575-44fb-b373-035534904600 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:57.939215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2e0d3-8575-44fb-b373-035534904600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:57.939454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2e0d3-8575-44fb-b373-035534904600 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:57.940198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2e0d3-8575-44fb-b373-035534904600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:57.942546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2e0d3-8575-44fb-b373-035534904600 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:58.050156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1253/2503] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:33:57 2022] POST /identity/v3/users => generated 299 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:33:58.054387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31e2ba-e2d2-4961-aecc-c8269d26c42f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:58.069298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31e2ba-e2d2-4961-aecc-c8269d26c42f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:58.074966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e7e27-bef9-41e3-aefb-9203b0392a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:58.076264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e7e27-bef9-41e3-aefb-9203b0392a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:58.076695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e7e27-bef9-41e3-aefb-9203b0392a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:58.077154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e7e27-bef9-41e3-aefb-9203b0392a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:58.078019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e7e27-bef9-41e3-aefb-9203b0392a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:58.080055 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e7e27-bef9-41e3-aefb-9203b0392a6f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:58.093090 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1251/2504] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:33:58.099719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2e0d3-8575-44fb-b373-035534904600 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:58.121876 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2e0d3-8575-44fb-b373-035534904600 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:58.136004 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4ea73-413c-4476-bb23-d66dbe0e6c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:58.136701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4ea73-413c-4476-bb23-d66dbe0e6c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:58.139360 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4ea73-413c-4476-bb23-d66dbe0e6c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:58.139678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4ea73-413c-4476-bb23-d66dbe0e6c54 admin admin] PATH_INFO: `/v3/projects/e37408e041064b5fac70c14c7bcb80e2/users/3470f45a9737400499300a2f3b34ebc7/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:58.163156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4ea73-413c-4476-bb23-d66dbe0e6c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e37408e041064b5fac70c14c7bcb80e2, user_id=3470f45a9737400499300a2f3b34ebc7, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:58.166103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4ea73-413c-4476-bb23-d66dbe0e6c54 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:58.262289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1254/2505] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:33:58 2022] PUT /identity/v3/projects/e37408e041064b5fac70c14c7bcb80e2/users/3470f45a9737400499300a2f3b34ebc7/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:33:58.266363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e7e27-bef9-41e3-aefb-9203b0392a6f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:58.282187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e7e27-bef9-41e3-aefb-9203b0392a6f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:58.376910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc68fa0-aa1b-4143-b534-e7d10103ac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:58.376910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc68fa0-aa1b-4143-b534-e7d10103ac87 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:58.376910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc68fa0-aa1b-4143-b534-e7d10103ac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:58.376910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc68fa0-aa1b-4143-b534-e7d10103ac87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:58.377885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc68fa0-aa1b-4143-b534-e7d10103ac87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:58.377885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc68fa0-aa1b-4143-b534-e7d10103ac87 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:58.389984 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1252/2506] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:33:58.393562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4ea73-413c-4476-bb23-d66dbe0e6c54 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:33:58.409968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4ea73-413c-4476-bb23-d66dbe0e6c54 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:33:58.414924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ff295-6d9f-4e7a-b74b-a99263d00dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:33:58.416049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ff295-6d9f-4e7a-b74b-a99263d00dff admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:33:58.416049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ff295-6d9f-4e7a-b74b-a99263d00dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:33:58.416268 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ff295-6d9f-4e7a-b74b-a99263d00dff admin admin] PATH_INFO: `/v3/projects/e37408e041064b5fac70c14c7bcb80e2/users/3470f45a9737400499300a2f3b34ebc7/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:33:58.418069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ff295-6d9f-4e7a-b74b-a99263d00dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e37408e041064b5fac70c14c7bcb80e2, user_id=3470f45a9737400499300a2f3b34ebc7, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:33:58.423457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ff295-6d9f-4e7a-b74b-a99263d00dff admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:33:58.432683 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1255/2507] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:33:58 2022] PUT /identity/v3/projects/e37408e041064b5fac70c14c7bcb80e2/users/3470f45a9737400499300a2f3b34ebc7/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:34:04.687580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374959bf-3a8c-4d9e-9cea-62a3261b5f6e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:04.688177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374959bf-3a8c-4d9e-9cea-62a3261b5f6e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:04.688337 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374959bf-3a8c-4d9e-9cea-62a3261b5f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:04.729570 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-374959bf-3a8c-4d9e-9cea-62a3261b5f6e None None] MFA Rules not processed for user `3470f45a9737400499300a2f3b34ebc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:34:04.730814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-374959bf-3a8c-4d9e-9cea-62a3261b5f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:04.819302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1253/2508] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:34:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:34:04.836013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ff295-6d9f-4e7a-b74b-a99263d00dff admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:04.836013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-620ff295-6d9f-4e7a-b74b-a99263d00dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:04.910800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ff295-6d9f-4e7a-b74b-a99263d00dff admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:04.914535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e64d-fefc-4621-8f87-953b1417ca6e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '237bbeebe2a949aba444353c1d2ee04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:04.915302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e64d-fefc-4621-8f87-953b1417ca6e service glance] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:04.915722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e64d-fefc-4621-8f87-953b1417ca6e service glance] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:04.915988 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e64d-fefc-4621-8f87-953b1417ca6e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:04.976084 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51e64d-fefc-4621-8f87-953b1417ca6e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:04.977945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51e64d-fefc-4621-8f87-953b1417ca6e service glance] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:04.993384 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1256/2509] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:05.260144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b07e5-4ae8-45da-a5e6-c9ef112f3c17 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:05.260435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b07e5-4ae8-45da-a5e6-c9ef112f3c17 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:05.260763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b07e5-4ae8-45da-a5e6-c9ef112f3c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:05.285072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d79b07e5-4ae8-45da-a5e6-c9ef112f3c17 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:34:05.286293 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d79b07e5-4ae8-45da-a5e6-c9ef112f3c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:05.305496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:05.346643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1254/2510] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:34:05.474743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e64d-fefc-4621-8f87-953b1417ca6e service glance] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:05.476234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8b51e64d-fefc-4621-8f87-953b1417ca6e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:05.551651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e64d-fefc-4621-8f87-953b1417ca6e service glance] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:05.557243 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81680b13-df49-4aa4-b32d-37036793e250 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:05.557942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81680b13-df49-4aa4-b32d-37036793e250 service swift] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:05.558225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81680b13-df49-4aa4-b32d-37036793e250 service swift] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:05.558522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81680b13-df49-4aa4-b32d-37036793e250 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:05.564272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:05.568501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81680b13-df49-4aa4-b32d-37036793e250 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:05.571506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81680b13-df49-4aa4-b32d-37036793e250 service swift] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:05.579946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1257/2511] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:34:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:34:10.831425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b07e5-4ae8-45da-a5e6-c9ef112f3c17 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:10.832987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d79b07e5-4ae8-45da-a5e6-c9ef112f3c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:10.907965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b07e5-4ae8-45da-a5e6-c9ef112f3c17 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:10.912169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d34a97-c63d-4e8a-ae80-091e928d248b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:10.912898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d34a97-c63d-4e8a-ae80-091e928d248b service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:10.913257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d34a97-c63d-4e8a-ae80-091e928d248b service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:10.913605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d34a97-c63d-4e8a-ae80-091e928d248b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:10.915759 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-45d34a97-c63d-4e8a-ae80-091e928d248b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:11.009641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d34a97-c63d-4e8a-ae80-091e928d248b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:11.011170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d34a97-c63d-4e8a-ae80-091e928d248b service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:11.025027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1255/2512] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:12.245925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81680b13-df49-4aa4-b32d-37036793e250 service swift] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:12.255723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81680b13-df49-4aa4-b32d-37036793e250 service swift] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:12.259359 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec5368-abb4-4fa8-b6a3-7d679abb925a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:12.259985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec5368-abb4-4fa8-b6a3-7d679abb925a service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:12.260264 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec5368-abb4-4fa8-b6a3-7d679abb925a service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:12.260537 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec5368-abb4-4fa8-b6a3-7d679abb925a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:12.262584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f0ec5368-abb4-4fa8-b6a3-7d679abb925a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:12.276900 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec5368-abb4-4fa8-b6a3-7d679abb925a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:12.281309 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec5368-abb4-4fa8-b6a3-7d679abb925a service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:12.290847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1258/2513] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:17.892028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5359547-2585-4c1f-8ac7-23ed277997a2 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:17.892585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5359547-2585-4c1f-8ac7-23ed277997a2 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:17.892585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5359547-2585-4c1f-8ac7-23ed277997a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:17.912893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b5359547-2585-4c1f-8ac7-23ed277997a2 None None] MFA Rules not processed for user `087ceea13e3747f49096b42d9d5c1c86`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:34:17.914100 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b5359547-2585-4c1f-8ac7-23ed277997a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:17.921739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1256/2514] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:34:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:34:22.285512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=9 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:23.110707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553d9ac-06f6-42dd-a9c7-e2cce7f31c1c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:23.111255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553d9ac-06f6-42dd-a9c7-e2cce7f31c1c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:23.111839 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553d9ac-06f6-42dd-a9c7-e2cce7f31c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:23.131792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-a553d9ac-06f6-42dd-a9c7-e2cce7f31c1c None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:34:23.132943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a553d9ac-06f6-42dd-a9c7-e2cce7f31c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:23.144374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1259/2515] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:34:23.157310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5359547-2585-4c1f-8ac7-23ed277997a2 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:23.170019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5359547-2585-4c1f-8ac7-23ed277997a2 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:23.175166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea826c-b1f3-4621-a858-050977919582 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:23.175806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea826c-b1f3-4621-a858-050977919582 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:23.176083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea826c-b1f3-4621-a858-050977919582 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:23.176354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea826c-b1f3-4621-a858-050977919582 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:23.185298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea826c-b1f3-4621-a858-050977919582 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:23.186902 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea826c-b1f3-4621-a858-050977919582 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:23.196022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1257/2516] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:34:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:34:25.793621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553d9ac-06f6-42dd-a9c7-e2cce7f31c1c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:25.799618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a553d9ac-06f6-42dd-a9c7-e2cce7f31c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:25.815703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553d9ac-06f6-42dd-a9c7-e2cce7f31c1c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:25.821183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a3206-b739-4eaf-b05c-b9ae622795ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:25.821786 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a3206-b739-4eaf-b05c-b9ae622795ea service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:25.822063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a3206-b739-4eaf-b05c-b9ae622795ea service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:25.822332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a3206-b739-4eaf-b05c-b9ae622795ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:25.824110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7e6a3206-b739-4eaf-b05c-b9ae622795ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:25.901437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a3206-b739-4eaf-b05c-b9ae622795ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:25.903152 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a3206-b739-4eaf-b05c-b9ae622795ea service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:25.914345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1260/2517] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:27.386052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea826c-b1f3-4621-a858-050977919582 service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:27.400153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea826c-b1f3-4621-a858-050977919582 service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:27.406036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e0d2c-3004-45f7-9b19-1e4e104c6c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:27.406649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e0d2c-3004-45f7-9b19-1e4e104c6c3a service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:27.406971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e0d2c-3004-45f7-9b19-1e4e104c6c3a service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:27.407789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e0d2c-3004-45f7-9b19-1e4e104c6c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:27.409598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-736e0d2c-3004-45f7-9b19-1e4e104c6c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:27.464218 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:27.478503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e0d2c-3004-45f7-9b19-1e4e104c6c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:27.480405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e0d2c-3004-45f7-9b19-1e4e104c6c3a service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:27.491477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1258/2518] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:35.909326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=31 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:37.517280 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:34:43.531057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a3206-b739-4eaf-b05c-b9ae622795ea service nova] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:43.541420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a3206-b739-4eaf-b05c-b9ae622795ea service nova] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:43.546858 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f6802-0cfa-4ceb-ba06-8b502486b161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:43.547879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f6802-0cfa-4ceb-ba06-8b502486b161 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:43.548463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f6802-0cfa-4ceb-ba06-8b502486b161 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:43.549511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f6802-0cfa-4ceb-ba06-8b502486b161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:43.551513 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f59f6802-0cfa-4ceb-ba06-8b502486b161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:43.569815 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f6802-0cfa-4ceb-ba06-8b502486b161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:43.571275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f6802-0cfa-4ceb-ba06-8b502486b161 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:43.582605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1261/2519] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:46.349929 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e0d2c-3004-45f7-9b19-1e4e104c6c3a service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:46.365534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e0d2c-3004-45f7-9b19-1e4e104c6c3a service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:46.372093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea6e6e-7873-4067-8462-637fe9326c5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:46.373107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea6e6e-7873-4067-8462-637fe9326c5b service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:46.373632 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea6e6e-7873-4067-8462-637fe9326c5b service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:46.373945 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea6e6e-7873-4067-8462-637fe9326c5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:46.376061 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2dea6e6e-7873-4067-8462-637fe9326c5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:46.405064 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea6e6e-7873-4067-8462-637fe9326c5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:46.406775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea6e6e-7873-4067-8462-637fe9326c5b service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:46.426350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1259/2520] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:49.546394 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250cf2c-f86e-4000-85a3-4b5051663238 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:49.546917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250cf2c-f86e-4000-85a3-4b5051663238 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:49.547043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250cf2c-f86e-4000-85a3-4b5051663238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:49.570476 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-4250cf2c-f86e-4000-85a3-4b5051663238 None None] MFA Rules not processed for user `e7a29212dfae4a93beea7d11085cde63`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:34:49.571724 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4250cf2c-f86e-4000-85a3-4b5051663238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:49.648293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1262/2521] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:34:50.034993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea6e6e-7873-4067-8462-637fe9326c5b service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:50.045377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea6e6e-7873-4067-8462-637fe9326c5b service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:50.054879 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b27ca-17a1-42dd-bc2c-92b3966568ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:50.055569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b27ca-17a1-42dd-bc2c-92b3966568ae service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:50.055899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b27ca-17a1-42dd-bc2c-92b3966568ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:50.056362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b27ca-17a1-42dd-bc2c-92b3966568ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:50.058374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-af3b27ca-17a1-42dd-bc2c-92b3966568ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:50.070074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b27ca-17a1-42dd-bc2c-92b3966568ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:50.071951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b27ca-17a1-42dd-bc2c-92b3966568ae service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:50.081889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1260/2522] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:51.005571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250cf2c-f86e-4000-85a3-4b5051663238 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:51.007529 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4250cf2c-f86e-4000-85a3-4b5051663238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:51.101179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250cf2c-f86e-4000-85a3-4b5051663238 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:51.106317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407eaf1c-9fd9-4e9d-8538-354c7818a309 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:51.106944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eaf1c-9fd9-4e9d-8538-354c7818a309 service placement] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:51.107215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eaf1c-9fd9-4e9d-8538-354c7818a309 service placement] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:51.107487 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eaf1c-9fd9-4e9d-8538-354c7818a309 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:51.109287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-407eaf1c-9fd9-4e9d-8538-354c7818a309 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:51.121604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407eaf1c-9fd9-4e9d-8538-354c7818a309 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:51.123368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407eaf1c-9fd9-4e9d-8538-354c7818a309 service placement] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:51.134552 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1263/2523] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:34:57.571728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b27ca-17a1-42dd-bc2c-92b3966568ae service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:34:57.592509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b27ca-17a1-42dd-bc2c-92b3966568ae service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:34:57.603187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d499ba-412a-424b-93f7-f43692b38a41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:34:57.603851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d499ba-412a-424b-93f7-f43692b38a41 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:34:57.604257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d499ba-412a-424b-93f7-f43692b38a41 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:34:57.604536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d499ba-412a-424b-93f7-f43692b38a41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:34:57.607426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e3d499ba-412a-424b-93f7-f43692b38a41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:34:57.681442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d499ba-412a-424b-93f7-f43692b38a41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:34:57.682941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d499ba-412a-424b-93f7-f43692b38a41 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:34:57.693585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1261/2524] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:34:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:01.133737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=63 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:01.713935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06579091-9d5c-4f46-aefe-b35a8b6af585 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:01.714218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06579091-9d5c-4f46-aefe-b35a8b6af585 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:01.714492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06579091-9d5c-4f46-aefe-b35a8b6af585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:01.733755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-06579091-9d5c-4f46-aefe-b35a8b6af585 None None] MFA Rules not processed for user `de2273220e824339bd13d968c75df26b`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:01.735056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-06579091-9d5c-4f46-aefe-b35a8b6af585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:01.742345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1264/2525] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:03.690905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d499ba-412a-424b-93f7-f43692b38a41 service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:03.701636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d499ba-412a-424b-93f7-f43692b38a41 service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:03.706683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ee294-5b95-4960-b712-828bab468333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:03.707776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ee294-5b95-4960-b712-828bab468333 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:03.708275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ee294-5b95-4960-b712-828bab468333 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:03.708739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ee294-5b95-4960-b712-828bab468333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:03.722883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ee294-5b95-4960-b712-828bab468333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:03.724986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ee294-5b95-4960-b712-828bab468333 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:03.757432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1262/2526] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:05.270209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06579091-9d5c-4f46-aefe-b35a8b6af585 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:05.283993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06579091-9d5c-4f46-aefe-b35a8b6af585 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:05.291627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cf7ad-fbf6-4458-bb3c-8b4a61231e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:05.292993 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cf7ad-fbf6-4458-bb3c-8b4a61231e10 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:05.293271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cf7ad-fbf6-4458-bb3c-8b4a61231e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:05.293540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cf7ad-fbf6-4458-bb3c-8b4a61231e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:05.295334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9e0cf7ad-fbf6-4458-bb3c-8b4a61231e10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:05.373820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cf7ad-fbf6-4458-bb3c-8b4a61231e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:05.375415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cf7ad-fbf6-4458-bb3c-8b4a61231e10 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:05.389140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1265/2527] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:10.743880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c775f78-ae76-42ba-80db-c748739e6ff4 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:10.744479 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c775f78-ae76-42ba-80db-c748739e6ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:10.744960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c775f78-ae76-42ba-80db-c748739e6ff4 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:10.746532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1263/2528] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:35:10.759125 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e92777-187b-44ac-acbc-be2618806748 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:10.759669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e92777-187b-44ac-acbc-be2618806748 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:10.774399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e92777-187b-44ac-acbc-be2618806748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:10.782577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-83e92777-187b-44ac-acbc-be2618806748 None None] MFA Rules not processed for user `90d65372febb414d82843f9dfa56f738`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:10.783907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-83e92777-187b-44ac-acbc-be2618806748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:10.793343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1266/2529] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:35:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:10.812261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c775f78-ae76-42ba-80db-c748739e6ff4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:10.821435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=47 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:10.833256 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c775f78-ae76-42ba-80db-c748739e6ff4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:10.845196 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c85585-edd8-410c-a7a0-976cdb78e0e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:10.846003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c85585-edd8-410c-a7a0-976cdb78e0e1 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:10.846389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c85585-edd8-410c-a7a0-976cdb78e0e1 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:10.846800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c85585-edd8-410c-a7a0-976cdb78e0e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:10.857038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c85585-edd8-410c-a7a0-976cdb78e0e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:10.858842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c85585-edd8-410c-a7a0-976cdb78e0e1 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:10.870200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1264/2530] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:10.980613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e92777-187b-44ac-acbc-be2618806748 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:10.980613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-83e92777-187b-44ac-acbc-be2618806748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:10.997650 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e92777-187b-44ac-acbc-be2618806748 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:11.008113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:11.008999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:11.009281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:11.009561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] PATH_INFO: `/v3/users/f5543fbd902d441cb6f03d3155f1af8d` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:11.010740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5543fbd902d441cb6f03d3155f1af8d)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:11.012878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:11.268622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5543fbd902d441cb6f03d3155f1af8d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:11.269840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5543fbd902d441cb6f03d3155f1af8d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:11.363101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5543fbd902d441cb6f03d3155f1af8d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:11.383835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5543fbd902d441cb6f03d3155f1af8d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:11.385117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2bb1549b-faaa-4acd-a367-01e4ae4227fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5543fbd902d441cb6f03d3155f1af8d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:11.410579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1267/2531] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:35:10 2022] DELETE /identity/v3/users/f5543fbd902d441cb6f03d3155f1af8d => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:11.684983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c85585-edd8-410c-a7a0-976cdb78e0e1 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:11.709636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c85585-edd8-410c-a7a0-976cdb78e0e1 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:11.788736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:11.790323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:11.790604 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:11.790960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] PATH_INFO: `/v3/projects/442bf64c97d945feb79f64cdd66baa28` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:11.792712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=442bf64c97d945feb79f64cdd66baa28)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:11.794906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:11.965232 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442bf64c97d945feb79f64cdd66baa28', 'request_id': 'req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:11.975697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442bf64c97d945feb79f64cdd66baa28', 'request_id': 'req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:12.071421 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442bf64c97d945feb79f64cdd66baa28', 'request_id': 'req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:12.091687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] The token cache is being invalidate because project 442bf64c97d945feb79f64cdd66baa28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:35:12.091968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:12.093378 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9dfe0ff3-df3e-449f-88b7-9f75a263c2ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:12.095044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1265/2532] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:35:11 2022] DELETE /identity/v3/projects/442bf64c97d945feb79f64cdd66baa28 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:12.103458 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434990ec-af77-4eb3-a3d0-786d8badac24 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:12.104020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434990ec-af77-4eb3-a3d0-786d8badac24 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:12.104020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434990ec-af77-4eb3-a3d0-786d8badac24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:12.128027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-434990ec-af77-4eb3-a3d0-786d8badac24 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:12.129348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-434990ec-af77-4eb3-a3d0-786d8badac24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:12.187150 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:12.217217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1268/2533] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:12.225520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5566a7-85b9-4d0e-ad77-a2c69a9544af None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:12.226052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5566a7-85b9-4d0e-ad77-a2c69a9544af None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:12.226564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5566a7-85b9-4d0e-ad77-a2c69a9544af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:12.265262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-de5566a7-85b9-4d0e-ad77-a2c69a9544af None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:12.270106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-de5566a7-85b9-4d0e-ad77-a2c69a9544af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:12.284040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1266/2534] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:12.288659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434990ec-af77-4eb3-a3d0-786d8badac24 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:12.303151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434990ec-af77-4eb3-a3d0-786d8badac24 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:12.311513 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebca17-6194-4865-ae05-67d06b8458c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:12.312501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebca17-6194-4865-ae05-67d06b8458c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:12.312946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebca17-6194-4865-ae05-67d06b8458c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:12.313378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebca17-6194-4865-ae05-67d06b8458c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:12.314546 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebca17-6194-4865-ae05-67d06b8458c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:35:12.315386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebca17-6194-4865-ae05-67d06b8458c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:12.317139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebca17-6194-4865-ae05-67d06b8458c2 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:12.331267 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1269/2535] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:35:12.337111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5566a7-85b9-4d0e-ad77-a2c69a9544af None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:12.348883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5566a7-85b9-4d0e-ad77-a2c69a9544af None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:12.356226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bce1a-6118-4e35-ab20-a832aea1c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:12.357282 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bce1a-6118-4e35-ab20-a832aea1c7ff admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:12.357729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bce1a-6118-4e35-ab20-a832aea1c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:12.358823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bce1a-6118-4e35-ab20-a832aea1c7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:12.359726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631bce1a-6118-4e35-ab20-a832aea1c7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:12.363165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631bce1a-6118-4e35-ab20-a832aea1c7ff admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:12.468353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1267/2536] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:35:12 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:35:12.492031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebca17-6194-4865-ae05-67d06b8458c2 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:12.505510 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebca17-6194-4865-ae05-67d06b8458c2 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:12.590967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82514535-eb60-48ee-bf58-7a8c27bc6846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:12.593241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82514535-eb60-48ee-bf58-7a8c27bc6846 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:12.593656 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82514535-eb60-48ee-bf58-7a8c27bc6846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:12.594032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82514535-eb60-48ee-bf58-7a8c27bc6846 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:12.594800 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82514535-eb60-48ee-bf58-7a8c27bc6846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:12.596969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82514535-eb60-48ee-bf58-7a8c27bc6846 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:12.685095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1270/2537] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:35:12 2022] POST /identity/v3/users => generated 283 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:35:12.689233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bce1a-6118-4e35-ab20-a832aea1c7ff admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:12.702082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bce1a-6118-4e35-ab20-a832aea1c7ff admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:12.710539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53a479-7d95-4340-9a02-f058c2c11e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:12.711172 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53a479-7d95-4340-9a02-f058c2c11e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:12.711449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53a479-7d95-4340-9a02-f058c2c11e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:12.711997 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53a479-7d95-4340-9a02-f058c2c11e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:12.712405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53a479-7d95-4340-9a02-f058c2c11e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:12.714036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53a479-7d95-4340-9a02-f058c2c11e31 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:12.728326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1268/2538] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:35:12.732217 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82514535-eb60-48ee-bf58-7a8c27bc6846 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:12.744215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82514535-eb60-48ee-bf58-7a8c27bc6846 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:12.750377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d0017-4dd6-4c7b-9f72-24f613b22114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:12.751251 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d0017-4dd6-4c7b-9f72-24f613b22114 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:12.751723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d0017-4dd6-4c7b-9f72-24f613b22114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:12.752813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d0017-4dd6-4c7b-9f72-24f613b22114 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:12.755595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d0017-4dd6-4c7b-9f72-24f613b22114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:12.755595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d0017-4dd6-4c7b-9f72-24f613b22114 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:12.770372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1271/2539] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:35:12.771957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53a479-7d95-4340-9a02-f058c2c11e31 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:12.784213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53a479-7d95-4340-9a02-f058c2c11e31 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:12.791064 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bb260-7777-4258-af17-ac3414f3847e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:12.791876 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bb260-7777-4258-af17-ac3414f3847e admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:12.792158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bb260-7777-4258-af17-ac3414f3847e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:12.792435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bb260-7777-4258-af17-ac3414f3847e admin admin] PATH_INFO: `/v3/projects/4ec98df514ca4e518c89fa1a9761f53f/users/2f6856049687424991db021bbcd434f2/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:12.810497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bb260-7777-4258-af17-ac3414f3847e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec98df514ca4e518c89fa1a9761f53f, user_id=2f6856049687424991db021bbcd434f2, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:12.815854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bb260-7777-4258-af17-ac3414f3847e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:12.879647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1269/2540] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:35:12 2022] PUT /identity/v3/projects/4ec98df514ca4e518c89fa1a9761f53f/users/2f6856049687424991db021bbcd434f2/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:35:12.891011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f594c-afab-42c6-84f0-b9a39332a2b0 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:12.891561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f594c-afab-42c6-84f0-b9a39332a2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:12.891883 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f594c-afab-42c6-84f0-b9a39332a2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:12.927704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ed1f594c-afab-42c6-84f0-b9a39332a2b0 None None] MFA Rules not processed for user `2f6856049687424991db021bbcd434f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:12.929003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ed1f594c-afab-42c6-84f0-b9a39332a2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:13.003808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1272/2541] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:13.015317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bb260-7777-4258-af17-ac3414f3847e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:13.017074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-392bb260-7777-4258-af17-ac3414f3847e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:13.044278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:13.087791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bb260-7777-4258-af17-ac3414f3847e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:13.091713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cf2ee-009b-474e-bc53-087d2f3ecf36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:13.094220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cf2ee-009b-474e-bc53-087d2f3ecf36 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:13.094220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cf2ee-009b-474e-bc53-087d2f3ecf36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:13.094220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cf2ee-009b-474e-bc53-087d2f3ecf36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:13.102805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cf2ee-009b-474e-bc53-087d2f3ecf36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:13.104482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cf2ee-009b-474e-bc53-087d2f3ecf36 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:13.115188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1270/2542] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:13.737169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f594c-afab-42c6-84f0-b9a39332a2b0 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:13.740238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ed1f594c-afab-42c6-84f0-b9a39332a2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:13.826480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:13.861020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f594c-afab-42c6-84f0-b9a39332a2b0 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:13.868989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c04ae-c5ca-48ba-b57d-4e4566e20634 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:13.869624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c04ae-c5ca-48ba-b57d-4e4566e20634 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:13.869902 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c04ae-c5ca-48ba-b57d-4e4566e20634 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:13.870176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c04ae-c5ca-48ba-b57d-4e4566e20634 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:13.872337 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-782c04ae-c5ca-48ba-b57d-4e4566e20634 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:13.904102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c04ae-c5ca-48ba-b57d-4e4566e20634 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:13.905873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c04ae-c5ca-48ba-b57d-4e4566e20634 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:13.920393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1273/2543] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:18.965470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cf2ee-009b-474e-bc53-087d2f3ecf36 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:18.992056 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cf2ee-009b-474e-bc53-087d2f3ecf36 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:19.000244 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9611e29-0536-4b7d-8868-3c73159453ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:19.001166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9611e29-0536-4b7d-8868-3c73159453ac service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:19.001820 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9611e29-0536-4b7d-8868-3c73159453ac service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:19.002106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9611e29-0536-4b7d-8868-3c73159453ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:19.004164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e9611e29-0536-4b7d-8868-3c73159453ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:19.025137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9611e29-0536-4b7d-8868-3c73159453ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:19.026743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9611e29-0536-4b7d-8868-3c73159453ac service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:19.040135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1271/2544] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:21.095343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99265133-df0f-4387-b59e-205bfa952341 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:21.096224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99265133-df0f-4387-b59e-205bfa952341 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:21.096224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99265133-df0f-4387-b59e-205bfa952341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:21.119361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-99265133-df0f-4387-b59e-205bfa952341 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:21.120709 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-99265133-df0f-4387-b59e-205bfa952341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:21.208693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1274/2545] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:35:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:21.235223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9611e29-0536-4b7d-8868-3c73159453ac service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:21.237412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e9611e29-0536-4b7d-8868-3c73159453ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:21.327909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9611e29-0536-4b7d-8868-3c73159453ac service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:21.333419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68babda-1861-4acc-ac5c-a702009338b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:21.334120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68babda-1861-4acc-ac5c-a702009338b7 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:21.334470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68babda-1861-4acc-ac5c-a702009338b7 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:21.334812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68babda-1861-4acc-ac5c-a702009338b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:21.345194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68babda-1861-4acc-ac5c-a702009338b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:21.347731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68babda-1861-4acc-ac5c-a702009338b7 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:21.356942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1272/2546] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:35:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 12:35:23.603868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99265133-df0f-4387-b59e-205bfa952341 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:23.778226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99265133-df0f-4387-b59e-205bfa952341 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:23.786177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:23.786837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:23.787116 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:23.787390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] PATH_INFO: `/v3/users/2f6856049687424991db021bbcd434f2` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:23.795363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f6856049687424991db021bbcd434f2)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:23.797400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:24.025999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f6856049687424991db021bbcd434f2'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:24.027111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f6856049687424991db021bbcd434f2'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:24.163198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f6856049687424991db021bbcd434f2'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:24.178257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f6856049687424991db021bbcd434f2'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:24.179080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f6856049687424991db021bbcd434f2'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:24.189351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:24.202000 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1275/2547] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:35:23 2022] DELETE /identity/v3/users/2f6856049687424991db021bbcd434f2 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:24.208559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20a356-6d04-49d6-a44d-366c046fd263 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:24.209576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20a356-6d04-49d6-a44d-366c046fd263 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:24.209576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20a356-6d04-49d6-a44d-366c046fd263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:24.230613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7c20a356-6d04-49d6-a44d-366c046fd263 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:24.236295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7c20a356-6d04-49d6-a44d-366c046fd263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:24.322965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1273/2548] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:24.340469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:24.342471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:24.417940 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1c483-eadb-4586-b7dc-a50bf9aff4ba admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:24.423416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7f21c-bdcd-4749-af8e-5e67cd5a79e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:24.424292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7f21c-bdcd-4749-af8e-5e67cd5a79e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:24.425181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7f21c-bdcd-4749-af8e-5e67cd5a79e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:24.425552 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7f21c-bdcd-4749-af8e-5e67cd5a79e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:24.436498 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7f21c-bdcd-4749-af8e-5e67cd5a79e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:24.437994 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7f21c-bdcd-4749-af8e-5e67cd5a79e3 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:24.450683 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1276/2549] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:35:25.167786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20a356-6d04-49d6-a44d-366c046fd263 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:25.169404 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7c20a356-6d04-49d6-a44d-366c046fd263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:25.183496 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20a356-6d04-49d6-a44d-366c046fd263 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:25.193001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:25.193758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:25.194021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:25.194372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] PATH_INFO: `/v3/projects/4ec98df514ca4e518c89fa1a9761f53f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:25.195497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ec98df514ca4e518c89fa1a9761f53f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:25.197555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:25.313252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:25.332586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ec98df514ca4e518c89fa1a9761f53f', 'request_id': 'req-ae1ac021-4848-42dd-a1fe-e42441f35d15'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:25.339175 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ec98df514ca4e518c89fa1a9761f53f', 'request_id': 'req-ae1ac021-4848-42dd-a1fe-e42441f35d15'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:25.403072 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ec98df514ca4e518c89fa1a9761f53f', 'request_id': 'req-ae1ac021-4848-42dd-a1fe-e42441f35d15'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:25.416757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] The token cache is being invalidate because project 4ec98df514ca4e518c89fa1a9761f53f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:35:25.417119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:25.417783 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ae1ac021-4848-42dd-a1fe-e42441f35d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:25.427727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e363c2-5f0b-47c2-864f-ba7817ef07ae None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:25.428353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e363c2-5f0b-47c2-864f-ba7817ef07ae None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:25.428353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e363c2-5f0b-47c2-864f-ba7817ef07ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:25.432323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1274/2550] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:35:25 2022] DELETE /identity/v3/projects/4ec98df514ca4e518c89fa1a9761f53f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:25.452093 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-57e363c2-5f0b-47c2-864f-ba7817ef07ae None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:25.453349 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-57e363c2-5f0b-47c2-864f-ba7817ef07ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:25.535244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1277/2551] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:25.544650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199fa26-e80e-42d1-9f20-c790d2a8b58b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:25.544994 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199fa26-e80e-42d1-9f20-c790d2a8b58b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:25.545199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199fa26-e80e-42d1-9f20-c790d2a8b58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:25.573614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1199fa26-e80e-42d1-9f20-c790d2a8b58b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:25.575207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1199fa26-e80e-42d1-9f20-c790d2a8b58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:25.591157 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1275/2552] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:25.595672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e363c2-5f0b-47c2-864f-ba7817ef07ae None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:25.610156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e363c2-5f0b-47c2-864f-ba7817ef07ae None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:25.618013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f1d73-6963-4dd7-b8ab-1737fdc5839f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:25.618642 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1d73-6963-4dd7-b8ab-1737fdc5839f admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:25.618924 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1d73-6963-4dd7-b8ab-1737fdc5839f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:25.619199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1d73-6963-4dd7-b8ab-1737fdc5839f admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:25.619901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f1d73-6963-4dd7-b8ab-1737fdc5839f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:35:25.620209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f1d73-6963-4dd7-b8ab-1737fdc5839f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:25.621828 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f1d73-6963-4dd7-b8ab-1737fdc5839f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:25.639187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1278/2553] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:35:25.644644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199fa26-e80e-42d1-9f20-c790d2a8b58b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:25.661026 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199fa26-e80e-42d1-9f20-c790d2a8b58b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:25.669036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5af3fa-db27-4ab6-8224-46de8925589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:25.669772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5af3fa-db27-4ab6-8224-46de8925589a admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:25.670105 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5af3fa-db27-4ab6-8224-46de8925589a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:25.670437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5af3fa-db27-4ab6-8224-46de8925589a admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:25.671299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5af3fa-db27-4ab6-8224-46de8925589a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:25.673479 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5af3fa-db27-4ab6-8224-46de8925589a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:25.803163 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1276/2554] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:35:25 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:35:25.804549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f1d73-6963-4dd7-b8ab-1737fdc5839f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:25.821983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f1d73-6963-4dd7-b8ab-1737fdc5839f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:25.920856 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a03fd3-2340-498c-bc26-02e9cb7ecce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:25.921393 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a03fd3-2340-498c-bc26-02e9cb7ecce4 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:25.921675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a03fd3-2340-498c-bc26-02e9cb7ecce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:25.921950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a03fd3-2340-498c-bc26-02e9cb7ecce4 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:25.922585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a03fd3-2340-498c-bc26-02e9cb7ecce4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:25.925142 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a03fd3-2340-498c-bc26-02e9cb7ecce4 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:26.034638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1279/2555] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:35:25 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:35:26.038946 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5af3fa-db27-4ab6-8224-46de8925589a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:26.061173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5af3fa-db27-4ab6-8224-46de8925589a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:26.070819 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f3231-427e-43c8-8978-e89922e8690c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:26.071590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f3231-427e-43c8-8978-e89922e8690c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:26.071954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f3231-427e-43c8-8978-e89922e8690c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:26.072291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f3231-427e-43c8-8978-e89922e8690c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:26.073165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f3231-427e-43c8-8978-e89922e8690c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:26.074984 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f3231-427e-43c8-8978-e89922e8690c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:26.095610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1277/2556] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:35:26.097424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a03fd3-2340-498c-bc26-02e9cb7ecce4 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:26.113598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a03fd3-2340-498c-bc26-02e9cb7ecce4 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:26.119517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d5241-3bfe-4bc0-939b-b03c8785965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:26.120188 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d5241-3bfe-4bc0-939b-b03c8785965d admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:26.120472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d5241-3bfe-4bc0-939b-b03c8785965d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:26.120749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d5241-3bfe-4bc0-939b-b03c8785965d admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:26.121408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d5241-3bfe-4bc0-939b-b03c8785965d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:26.123165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d5241-3bfe-4bc0-939b-b03c8785965d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:26.149645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1280/2557] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:35:26.151178 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f3231-427e-43c8-8978-e89922e8690c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:26.167833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f3231-427e-43c8-8978-e89922e8690c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:26.179440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67491d72-00a6-4956-91e9-18fdc223da2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:26.181773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67491d72-00a6-4956-91e9-18fdc223da2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:26.182371 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67491d72-00a6-4956-91e9-18fdc223da2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:26.183518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67491d72-00a6-4956-91e9-18fdc223da2f admin admin] PATH_INFO: `/v3/projects/1f0014351fc146cfa99ea2ea8bf507a8/users/ca6fa9e1f7bc44fab8ac0c2fc7fdff1c/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:26.206068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67491d72-00a6-4956-91e9-18fdc223da2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0014351fc146cfa99ea2ea8bf507a8, user_id=ca6fa9e1f7bc44fab8ac0c2fc7fdff1c, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:26.209441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67491d72-00a6-4956-91e9-18fdc223da2f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:26.303584 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1278/2558] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:35:26 2022] PUT /identity/v3/projects/1f0014351fc146cfa99ea2ea8bf507a8/users/ca6fa9e1f7bc44fab8ac0c2fc7fdff1c/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:26.315145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c0822-11f1-49e3-81f4-96d88264c1c7 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:26.319809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c0822-11f1-49e3-81f4-96d88264c1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:26.319809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c0822-11f1-49e3-81f4-96d88264c1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:26.331232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:26.350640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ba4c0822-11f1-49e3-81f4-96d88264c1c7 None None] MFA Rules not processed for user `ca6fa9e1f7bc44fab8ac0c2fc7fdff1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:26.352088 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ba4c0822-11f1-49e3-81f4-96d88264c1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:26.443755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1281/2559] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:26.451129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67491d72-00a6-4956-91e9-18fdc223da2f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:26.452983 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-67491d72-00a6-4956-91e9-18fdc223da2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:26.564734 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67491d72-00a6-4956-91e9-18fdc223da2f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:26.569900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2667c-73e9-448f-901e-034546c61fd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:26.570995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2667c-73e9-448f-901e-034546c61fd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:26.571567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2667c-73e9-448f-901e-034546c61fd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:26.572418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2667c-73e9-448f-901e-034546c61fd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:26.583290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2667c-73e9-448f-901e-034546c61fd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:26.585052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2667c-73e9-448f-901e-034546c61fd2 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:26.595647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1279/2560] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:26.624093 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c0822-11f1-49e3-81f4-96d88264c1c7 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:26.717137 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c0822-11f1-49e3-81f4-96d88264c1c7 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:26.724722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:26.725534 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:26.725814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:26.726086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] PATH_INFO: `/v3/users/ca6fa9e1f7bc44fab8ac0c2fc7fdff1c` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:26.727274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6fa9e1f7bc44fab8ac0c2fc7fdff1c)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:26.729350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:26.977011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6fa9e1f7bc44fab8ac0c2fc7fdff1c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:26.977761 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6fa9e1f7bc44fab8ac0c2fc7fdff1c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:27.056975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6fa9e1f7bc44fab8ac0c2fc7fdff1c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:27.078300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6fa9e1f7bc44fab8ac0c2fc7fdff1c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:27.078930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6fa9e1f7bc44fab8ac0c2fc7fdff1c'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:27.100343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1282/2561] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:35:26 2022] DELETE /identity/v3/users/ca6fa9e1f7bc44fab8ac0c2fc7fdff1c => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:27.107152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e88c5-fbdf-4476-868d-730b93295e40 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:27.107447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e88c5-fbdf-4476-868d-730b93295e40 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:27.107988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e88c5-fbdf-4476-868d-730b93295e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:27.137109 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1f1e88c5-fbdf-4476-868d-730b93295e40 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:27.138636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1f1e88c5-fbdf-4476-868d-730b93295e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:27.218482 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:27.234114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1280/2562] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:27.245031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:27.246063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:27.272178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:27.370455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bc484-41aa-40e1-adbe-f5e3b1012fd8 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:27.381793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2729c2-b95e-4bca-be6c-8c097479ea54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:27.382785 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2729c2-b95e-4bca-be6c-8c097479ea54 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:27.384291 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2729c2-b95e-4bca-be6c-8c097479ea54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:27.385044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2729c2-b95e-4bca-be6c-8c097479ea54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:27.411901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2729c2-b95e-4bca-be6c-8c097479ea54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:27.413843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2729c2-b95e-4bca-be6c-8c097479ea54 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:27.437930 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1283/2563] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:28.254541 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e88c5-fbdf-4476-868d-730b93295e40 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:28.254541 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1f1e88c5-fbdf-4476-868d-730b93295e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:28.262503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e88c5-fbdf-4476-868d-730b93295e40 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:28.269195 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:28.269875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:28.270155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:28.270430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] PATH_INFO: `/v3/projects/1f0014351fc146cfa99ea2ea8bf507a8` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:28.271612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f0014351fc146cfa99ea2ea8bf507a8)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:28.273592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:28.431584 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f0014351fc146cfa99ea2ea8bf507a8', 'request_id': 'req-77203b7b-229a-446e-a50c-55cbba7064c0'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:28.437332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f0014351fc146cfa99ea2ea8bf507a8', 'request_id': 'req-77203b7b-229a-446e-a50c-55cbba7064c0'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:28.485619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f0014351fc146cfa99ea2ea8bf507a8', 'request_id': 'req-77203b7b-229a-446e-a50c-55cbba7064c0'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:28.512013 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] The token cache is being invalidate because project 1f0014351fc146cfa99ea2ea8bf507a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:35:28.512247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:28.514114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-77203b7b-229a-446e-a50c-55cbba7064c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:28.516264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1281/2564] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:35:28 2022] DELETE /identity/v3/projects/1f0014351fc146cfa99ea2ea8bf507a8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:28.521958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067ec3d-41f6-4640-b54b-eca1e8c6abff None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:28.522528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067ec3d-41f6-4640-b54b-eca1e8c6abff None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:28.522528 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067ec3d-41f6-4640-b54b-eca1e8c6abff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:28.548474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d067ec3d-41f6-4640-b54b-eca1e8c6abff None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:28.550723 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d067ec3d-41f6-4640-b54b-eca1e8c6abff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:28.627151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1284/2565] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:28.637088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a3807-e42e-418a-ac8f-b99e4b5287d1 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:28.637316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a3807-e42e-418a-ac8f-b99e4b5287d1 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:28.637584 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a3807-e42e-418a-ac8f-b99e4b5287d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:28.662234 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bf5a3807-e42e-418a-ac8f-b99e4b5287d1 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:28.663467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bf5a3807-e42e-418a-ac8f-b99e4b5287d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:28.691437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1282/2566] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:28.697282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067ec3d-41f6-4640-b54b-eca1e8c6abff None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:28.709646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067ec3d-41f6-4640-b54b-eca1e8c6abff None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:28.714583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48bcdf-933e-47a5-ad3f-fa56c6ebd818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:28.715574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48bcdf-933e-47a5-ad3f-fa56c6ebd818 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:28.715574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48bcdf-933e-47a5-ad3f-fa56c6ebd818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:28.715969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48bcdf-933e-47a5-ad3f-fa56c6ebd818 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:28.719329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48bcdf-933e-47a5-ad3f-fa56c6ebd818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:35:28.719329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48bcdf-933e-47a5-ad3f-fa56c6ebd818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:28.719329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48bcdf-933e-47a5-ad3f-fa56c6ebd818 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:28.739667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1285/2567] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:35:28.740770 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a3807-e42e-418a-ac8f-b99e4b5287d1 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:28.751208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a3807-e42e-418a-ac8f-b99e4b5287d1 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:28.757289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2c327-cd67-4008-9218-89402847dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:28.757934 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2c327-cd67-4008-9218-89402847dc3d admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:28.758301 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2c327-cd67-4008-9218-89402847dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:28.758576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2c327-cd67-4008-9218-89402847dc3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:28.759323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2c327-cd67-4008-9218-89402847dc3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:28.761419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2c327-cd67-4008-9218-89402847dc3d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:28.858590 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1283/2568] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:35:28 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:35:28.864838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48bcdf-933e-47a5-ad3f-fa56c6ebd818 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:28.898058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48bcdf-933e-47a5-ad3f-fa56c6ebd818 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:28.997576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c42b11-7320-404f-aca6-9f3747cca2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:28.998187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c42b11-7320-404f-aca6-9f3747cca2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:28.998473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c42b11-7320-404f-aca6-9f3747cca2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:28.998743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c42b11-7320-404f-aca6-9f3747cca2f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:28.999414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c42b11-7320-404f-aca6-9f3747cca2f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:29.001452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c42b11-7320-404f-aca6-9f3747cca2f3 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:29.089155 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1286/2569] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:35:28 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:35:29.093165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2c327-cd67-4008-9218-89402847dc3d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:29.108166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2c327-cd67-4008-9218-89402847dc3d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:29.113489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250207c-2edd-4c19-a5fc-5619609a2b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:29.114360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250207c-2edd-4c19-a5fc-5619609a2b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:29.114642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250207c-2edd-4c19-a5fc-5619609a2b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:29.114915 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250207c-2edd-4c19-a5fc-5619609a2b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:29.115608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250207c-2edd-4c19-a5fc-5619609a2b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:29.120909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250207c-2edd-4c19-a5fc-5619609a2b78 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:29.134505 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1284/2570] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:35:29.138549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c42b11-7320-404f-aca6-9f3747cca2f3 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:29.151308 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c42b11-7320-404f-aca6-9f3747cca2f3 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:29.157286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf313a2-f668-4f66-95cf-b4822efbf62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:29.158139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf313a2-f668-4f66-95cf-b4822efbf62a admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:29.158479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf313a2-f668-4f66-95cf-b4822efbf62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:29.158881 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf313a2-f668-4f66-95cf-b4822efbf62a admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:29.159672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf313a2-f668-4f66-95cf-b4822efbf62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:29.162734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf313a2-f668-4f66-95cf-b4822efbf62a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:29.178212 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1287/2571] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:35:29.182440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250207c-2edd-4c19-a5fc-5619609a2b78 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:29.199497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250207c-2edd-4c19-a5fc-5619609a2b78 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:29.204510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f8ed8-392d-41d9-840b-2262ccbdac61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:29.205208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f8ed8-392d-41d9-840b-2262ccbdac61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:29.205489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f8ed8-392d-41d9-840b-2262ccbdac61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:29.205850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f8ed8-392d-41d9-840b-2262ccbdac61 admin admin] PATH_INFO: `/v3/projects/c2d7016631054758b4434dbc575cd73c/users/75a9e3fc8f35436c9b5239c3e516dd13/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:29.227564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f8ed8-392d-41d9-840b-2262ccbdac61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d7016631054758b4434dbc575cd73c, user_id=75a9e3fc8f35436c9b5239c3e516dd13, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:29.230804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f8ed8-392d-41d9-840b-2262ccbdac61 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:29.327862 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1285/2572] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:35:29 2022] PUT /identity/v3/projects/c2d7016631054758b4434dbc575cd73c/users/75a9e3fc8f35436c9b5239c3e516dd13/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:29.328926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ca3c4-a77c-4f3e-ae9e-e7bb20467730 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:29.328926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ca3c4-a77c-4f3e-ae9e-e7bb20467730 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:29.329223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ca3c4-a77c-4f3e-ae9e-e7bb20467730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:29.361514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-b46ca3c4-a77c-4f3e-ae9e-e7bb20467730 None None] MFA Rules not processed for user `75a9e3fc8f35436c9b5239c3e516dd13`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:29.363745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b46ca3c4-a77c-4f3e-ae9e-e7bb20467730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:29.379967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:29.465016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1288/2573] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:29.478438 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f8ed8-392d-41d9-840b-2262ccbdac61 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:29.482362 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-3a8f8ed8-392d-41d9-840b-2262ccbdac61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:29.557776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f8ed8-392d-41d9-840b-2262ccbdac61 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:29.563085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94c31-266f-4398-a2d8-1135b4e5b892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:29.563085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94c31-266f-4398-a2d8-1135b4e5b892 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:29.563085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94c31-266f-4398-a2d8-1135b4e5b892 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:29.563507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94c31-266f-4398-a2d8-1135b4e5b892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:29.575318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b94c31-266f-4398-a2d8-1135b4e5b892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:29.577077 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b94c31-266f-4398-a2d8-1135b4e5b892 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:29.587385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1286/2574] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:29.628404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ca3c4-a77c-4f3e-ae9e-e7bb20467730 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:29.725078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ca3c4-a77c-4f3e-ae9e-e7bb20467730 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:29.730560 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:29.731411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:29.731838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:29.732111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] PATH_INFO: `/v3/users/75a9e3fc8f35436c9b5239c3e516dd13` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:29.733483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a9e3fc8f35436c9b5239c3e516dd13)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:29.735432 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:29.920688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a9e3fc8f35436c9b5239c3e516dd13'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:29.921461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a9e3fc8f35436c9b5239c3e516dd13'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:29.992545 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a9e3fc8f35436c9b5239c3e516dd13'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:30.009495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a9e3fc8f35436c9b5239c3e516dd13'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:30.010375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a9e3fc8f35436c9b5239c3e516dd13'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:30.037671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1289/2575] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:35:29 2022] DELETE /identity/v3/users/75a9e3fc8f35436c9b5239c3e516dd13 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:30.040016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018e7f9-3adc-468b-a109-9e42eb4c792b None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:30.040016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018e7f9-3adc-468b-a109-9e42eb4c792b None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:30.040208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018e7f9-3adc-468b-a109-9e42eb4c792b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:30.065369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d018e7f9-3adc-468b-a109-9e42eb4c792b None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:30.066825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d018e7f9-3adc-468b-a109-9e42eb4c792b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:30.085920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:30.179029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1287/2576] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:30.188606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:30.190097 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:30.239757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:30.271696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08951397-37fb-44a3-946e-66f222bad8a5 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:30.275502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075bc61-daee-46a2-ad44-062dd38b0bda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:30.276177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075bc61-daee-46a2-ad44-062dd38b0bda service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:30.276454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075bc61-daee-46a2-ad44-062dd38b0bda service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:30.276740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075bc61-daee-46a2-ad44-062dd38b0bda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:30.292814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075bc61-daee-46a2-ad44-062dd38b0bda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:30.296148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075bc61-daee-46a2-ad44-062dd38b0bda service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:30.315684 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1290/2577] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:30.834932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018e7f9-3adc-468b-a109-9e42eb4c792b None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:30.836721 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d018e7f9-3adc-468b-a109-9e42eb4c792b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:30.854709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018e7f9-3adc-468b-a109-9e42eb4c792b None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:30.862778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:30.863447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:30.864776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:30.865058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] PATH_INFO: `/v3/projects/c2d7016631054758b4434dbc575cd73c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:30.867157 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d7016631054758b4434dbc575cd73c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:30.869186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:30.986254 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d7016631054758b4434dbc575cd73c', 'request_id': 'req-a9bd611c-a615-498e-a52a-37cb3fef0121'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:30.997679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d7016631054758b4434dbc575cd73c', 'request_id': 'req-a9bd611c-a615-498e-a52a-37cb3fef0121'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:31.078287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d7016631054758b4434dbc575cd73c', 'request_id': 'req-a9bd611c-a615-498e-a52a-37cb3fef0121'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:31.092487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] The token cache is being invalidate because project c2d7016631054758b4434dbc575cd73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:35:31.092778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:31.097235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a9bd611c-a615-498e-a52a-37cb3fef0121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:31.097235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1288/2578] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:35:30 2022] DELETE /identity/v3/projects/c2d7016631054758b4434dbc575cd73c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:31.101491 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771c79c-925d-4618-ae1e-f97e08e7eb03 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:31.102029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771c79c-925d-4618-ae1e-f97e08e7eb03 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:31.102230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771c79c-925d-4618-ae1e-f97e08e7eb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:31.123727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-2771c79c-925d-4618-ae1e-f97e08e7eb03 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:31.124958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2771c79c-925d-4618-ae1e-f97e08e7eb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:31.206277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1291/2579] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:31.214634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce26bec-511a-4179-a209-4c41b044c87c None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:31.214954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce26bec-511a-4179-a209-4c41b044c87c None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:31.215198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce26bec-511a-4179-a209-4c41b044c87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:31.247508 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7ce26bec-511a-4179-a209-4c41b044c87c None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:31.248814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7ce26bec-511a-4179-a209-4c41b044c87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:31.263587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1289/2580] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:31.275591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771c79c-925d-4618-ae1e-f97e08e7eb03 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:31.279453 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771c79c-925d-4618-ae1e-f97e08e7eb03 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:31.285909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88c84e-08a3-4ec2-8de1-2e592251112c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:31.286659 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88c84e-08a3-4ec2-8de1-2e592251112c admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:31.287048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88c84e-08a3-4ec2-8de1-2e592251112c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:31.287479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88c84e-08a3-4ec2-8de1-2e592251112c admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:31.288378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88c84e-08a3-4ec2-8de1-2e592251112c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:35:31.288791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88c84e-08a3-4ec2-8de1-2e592251112c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:31.290539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88c84e-08a3-4ec2-8de1-2e592251112c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:31.306346 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1292/2581] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:35:31.315336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce26bec-511a-4179-a209-4c41b044c87c None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:31.327639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce26bec-511a-4179-a209-4c41b044c87c None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:31.335856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcd66c-0320-4882-a662-eef010d6e37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:31.336582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcd66c-0320-4882-a662-eef010d6e37e admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:31.336951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcd66c-0320-4882-a662-eef010d6e37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:31.337326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcd66c-0320-4882-a662-eef010d6e37e admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:31.338112 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcd66c-0320-4882-a662-eef010d6e37e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:31.340561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcd66c-0320-4882-a662-eef010d6e37e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:32.011287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1290/2582] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:35:31 2022] POST /identity/v3/projects => generated 397 bytes in 696 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:35:32.015237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88c84e-08a3-4ec2-8de1-2e592251112c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:32.025618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88c84e-08a3-4ec2-8de1-2e592251112c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:32.087498 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246a4b5-8880-43ea-9c44-5c5d244d9f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:32.088317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246a4b5-8880-43ea-9c44-5c5d244d9f74 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:32.088595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246a4b5-8880-43ea-9c44-5c5d244d9f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:32.088943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246a4b5-8880-43ea-9c44-5c5d244d9f74 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:32.090138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246a4b5-8880-43ea-9c44-5c5d244d9f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:32.092934 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246a4b5-8880-43ea-9c44-5c5d244d9f74 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:32.214819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1293/2583] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:35:32 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:35:32.219339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcd66c-0320-4882-a662-eef010d6e37e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:32.231087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcd66c-0320-4882-a662-eef010d6e37e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:32.238742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468536b4-770c-4000-a6b0-a7dce053df2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:32.239670 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468536b4-770c-4000-a6b0-a7dce053df2b admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:32.239966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468536b4-770c-4000-a6b0-a7dce053df2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:32.240214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468536b4-770c-4000-a6b0-a7dce053df2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:32.240865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468536b4-770c-4000-a6b0-a7dce053df2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:32.242660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468536b4-770c-4000-a6b0-a7dce053df2b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:32.261640 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246a4b5-8880-43ea-9c44-5c5d244d9f74 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:32.262074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1291/2584] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:35:32.273003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246a4b5-8880-43ea-9c44-5c5d244d9f74 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:32.278570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac173509-e2a9-462d-bb1c-28b2fc278f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:32.279573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac173509-e2a9-462d-bb1c-28b2fc278f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:32.279573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac173509-e2a9-462d-bb1c-28b2fc278f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:32.279791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac173509-e2a9-462d-bb1c-28b2fc278f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:32.280446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac173509-e2a9-462d-bb1c-28b2fc278f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:32.282080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac173509-e2a9-462d-bb1c-28b2fc278f41 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:32.297855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1294/2585] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:35:32.302511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468536b4-770c-4000-a6b0-a7dce053df2b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:32.315593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468536b4-770c-4000-a6b0-a7dce053df2b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:32.318689 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd5c55-efd9-476c-bd1b-fede9102180c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:32.319368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd5c55-efd9-476c-bd1b-fede9102180c admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:32.319695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd5c55-efd9-476c-bd1b-fede9102180c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:32.319982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd5c55-efd9-476c-bd1b-fede9102180c admin admin] PATH_INFO: `/v3/projects/5707d692130b40f298f0d122edfeed38/users/3cb1d1e3e4884e4ebbba8ab63ab7bfef/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:32.337651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd5c55-efd9-476c-bd1b-fede9102180c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5707d692130b40f298f0d122edfeed38, user_id=3cb1d1e3e4884e4ebbba8ab63ab7bfef, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:32.340352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd5c55-efd9-476c-bd1b-fede9102180c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:32.404810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1292/2586] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:35:32 2022] PUT /identity/v3/projects/5707d692130b40f298f0d122edfeed38/users/3cb1d1e3e4884e4ebbba8ab63ab7bfef/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:35:32.421595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a109-4b9a-4e03-918e-638f3dc5a05a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:32.421877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a109-4b9a-4e03-918e-638f3dc5a05a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:32.422151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a109-4b9a-4e03-918e-638f3dc5a05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:32.467864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-d429a109-4b9a-4e03-918e-638f3dc5a05a None None] MFA Rules not processed for user `3cb1d1e3e4884e4ebbba8ab63ab7bfef`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:32.469257 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-d429a109-4b9a-4e03-918e-638f3dc5a05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:32.515966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:32.571576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1295/2587] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:32.589953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd5c55-efd9-476c-bd1b-fede9102180c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:32.592953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4cfd5c55-efd9-476c-bd1b-fede9102180c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:32.684407 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:32.701137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd5c55-efd9-476c-bd1b-fede9102180c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:32.704954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152470b-e764-4e8f-8e79-b3f1a7d19032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:32.705701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152470b-e764-4e8f-8e79-b3f1a7d19032 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:32.706580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152470b-e764-4e8f-8e79-b3f1a7d19032 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:32.706867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152470b-e764-4e8f-8e79-b3f1a7d19032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:32.727300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152470b-e764-4e8f-8e79-b3f1a7d19032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:32.728790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152470b-e764-4e8f-8e79-b3f1a7d19032 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:32.740401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1293/2588] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:35:41.670778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a109-4b9a-4e03-918e-638f3dc5a05a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:41.738155 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a109-4b9a-4e03-918e-638f3dc5a05a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:41.743373 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:41.744063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:41.744356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:41.744633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] PATH_INFO: `/v3/users/3cb1d1e3e4884e4ebbba8ab63ab7bfef` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:41.745813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb1d1e3e4884e4ebbba8ab63ab7bfef)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:41.747868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:41.948301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb1d1e3e4884e4ebbba8ab63ab7bfef'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:41.948891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb1d1e3e4884e4ebbba8ab63ab7bfef'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:42.026777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb1d1e3e4884e4ebbba8ab63ab7bfef'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:42.049480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb1d1e3e4884e4ebbba8ab63ab7bfef'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:42.050898 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb1d1e3e4884e4ebbba8ab63ab7bfef'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:42.065045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1296/2589] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:35:41 2022] DELETE /identity/v3/users/3cb1d1e3e4884e4ebbba8ab63ab7bfef => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:42.071761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e87ae-d5a8-43e4-b00f-f654f7bbccb6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:42.072314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e87ae-d5a8-43e4-b00f-f654f7bbccb6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:42.072314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e87ae-d5a8-43e4-b00f-f654f7bbccb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:42.091967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-832e87ae-d5a8-43e4-b00f-f654f7bbccb6 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:42.093193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-832e87ae-d5a8-43e4-b00f-f654f7bbccb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:42.170982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1294/2590] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:42.181550 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:42.182970 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:42.247272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62304b1b-92d3-4590-870d-c5c8f1855676 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:42.250913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b15e4-f024-4c7c-9146-f30f57502179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:42.251587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b15e4-f024-4c7c-9146-f30f57502179 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:42.251875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b15e4-f024-4c7c-9146-f30f57502179 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:42.252144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b15e4-f024-4c7c-9146-f30f57502179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:42.258849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:42.264565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b15e4-f024-4c7c-9146-f30f57502179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:42.264565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b15e4-f024-4c7c-9146-f30f57502179 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:42.273836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1297/2591] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:35:42.827611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e87ae-d5a8-43e4-b00f-f654f7bbccb6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:42.830174 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-832e87ae-d5a8-43e4-b00f-f654f7bbccb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:42.848410 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e87ae-d5a8-43e4-b00f-f654f7bbccb6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:42.853826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:42.856247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:42.856465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:42.856743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] PATH_INFO: `/v3/projects/5707d692130b40f298f0d122edfeed38` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:42.857965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5707d692130b40f298f0d122edfeed38)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:42.859987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:43.013565 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5707d692130b40f298f0d122edfeed38', 'request_id': 'req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:43.019882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5707d692130b40f298f0d122edfeed38', 'request_id': 'req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:43.101306 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5707d692130b40f298f0d122edfeed38', 'request_id': 'req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:43.115104 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] The token cache is being invalidate because project 5707d692130b40f298f0d122edfeed38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:35:43.115391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:43.116188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:35:43.118509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1295/2592] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:35:42 2022] DELETE /identity/v3/projects/5707d692130b40f298f0d122edfeed38 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:35:46.135198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712c5b0-5cbb-49ca-bb18-5a0e27d52912 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:46.135739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712c5b0-5cbb-49ca-bb18-5a0e27d52912 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:46.135897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712c5b0-5cbb-49ca-bb18-5a0e27d52912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:46.157682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-b712c5b0-5cbb-49ca-bb18-5a0e27d52912 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:35:46.158913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b712c5b0-5cbb-49ca-bb18-5a0e27d52912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:46.216571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1298/2593] 10.222.0.62 () {58 vars in 1009 bytes} [Mon Sep 5 12:35:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:35:46.229229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:35:46.230693 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:35:46.302416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddd2be-32c5-438d-a6e1-2ba33a9fa468 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:35:46.307601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721ed5c-3294-45b8-a1df-d673e872fbb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:35:46.308214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721ed5c-3294-45b8-a1df-d673e872fbb3 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:35:46.308493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721ed5c-3294-45b8-a1df-d673e872fbb3 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:35:46.308763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721ed5c-3294-45b8-a1df-d673e872fbb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:35:46.314947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:46.317339 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721ed5c-3294-45b8-a1df-d673e872fbb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:35:46.318977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721ed5c-3294-45b8-a1df-d673e872fbb3 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:35:46.327368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1296/2594] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:35:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:35:56.209201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=28 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:35:56.512014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:01.346907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf9db3-c474-4a73-afae-727fd404feaf None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:01.347492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf9db3-c474-4a73-afae-727fd404feaf None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:01.347492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf9db3-c474-4a73-afae-727fd404feaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:01.367570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-9caf9db3-c474-4a73-afae-727fd404feaf None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:36:01.368783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9caf9db3-c474-4a73-afae-727fd404feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:01.377104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1299/2595] 10.222.0.62 () {58 vars in 967 bytes} [Mon Sep 5 12:36:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:36:01.390262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721ed5c-3294-45b8-a1df-d673e872fbb3 service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:01.404144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721ed5c-3294-45b8-a1df-d673e872fbb3 service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:01.409941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a6d39-aea4-4de5-95f7-1f86c6b8af78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:01.410557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a6d39-aea4-4de5-95f7-1f86c6b8af78 service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:01.410831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a6d39-aea4-4de5-95f7-1f86c6b8af78 service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:01.411101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a6d39-aea4-4de5-95f7-1f86c6b8af78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:01.420746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a6d39-aea4-4de5-95f7-1f86c6b8af78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:01.422318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a6d39-aea4-4de5-95f7-1f86c6b8af78 service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:01.430962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1297/2596] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:36:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 12:36:11.361727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:11.427371 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:27.439153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf9db3-c474-4a73-afae-727fd404feaf None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:27.439153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-9caf9db3-c474-4a73-afae-727fd404feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:27.504950 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf9db3-c474-4a73-afae-727fd404feaf None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:27.508597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9592e-367d-44af-8c56-5047917fc188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:27.509208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9592e-367d-44af-8c56-5047917fc188 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:27.509486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9592e-367d-44af-8c56-5047917fc188 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:27.509755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9592e-367d-44af-8c56-5047917fc188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:27.511664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-2be9592e-367d-44af-8c56-5047917fc188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:27.583463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9592e-367d-44af-8c56-5047917fc188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:27.585147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9592e-367d-44af-8c56-5047917fc188 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:27.596233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1300/2597] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:36:31.676053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a6d39-aea4-4de5-95f7-1f86c6b8af78 service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:31.677849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-063a6d39-aea4-4de5-95f7-1f86c6b8af78 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:31.695602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a6d39-aea4-4de5-95f7-1f86c6b8af78 service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:31.705892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:31.705892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:31.705892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:31.705892 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] PATH_INFO: `/v3/users/de2273220e824339bd13d968c75df26b` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:31.706271 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2273220e824339bd13d968c75df26b)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:31.708991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:31.900432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2273220e824339bd13d968c75df26b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:31.901091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2273220e824339bd13d968c75df26b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:31.982186 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2273220e824339bd13d968c75df26b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:31.996030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2273220e824339bd13d968c75df26b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:31.996749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1b9b0294-3597-4953-a877-7044b55595a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2273220e824339bd13d968c75df26b'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:32.009993 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1298/2598] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:36:31 2022] DELETE /identity/v3/users/de2273220e824339bd13d968c75df26b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:32.280842 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9592e-367d-44af-8c56-5047917fc188 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:32.295606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9592e-367d-44af-8c56-5047917fc188 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:32.356381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:32.357006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:32.357281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:32.357672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] PATH_INFO: `/v3/projects/4f6830d2e75a47c8a757d4a66c6695ca` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:32.358822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f6830d2e75a47c8a757d4a66c6695ca)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:32.360757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:32.481195 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f6830d2e75a47c8a757d4a66c6695ca', 'request_id': 'req-1986931f-f57c-4f7e-ac20-59cdc9028259'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:32.483295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:32.487813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f6830d2e75a47c8a757d4a66c6695ca', 'request_id': 'req-1986931f-f57c-4f7e-ac20-59cdc9028259'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:32.546752 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f6830d2e75a47c8a757d4a66c6695ca', 'request_id': 'req-1986931f-f57c-4f7e-ac20-59cdc9028259'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:32.559807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] The token cache is being invalidate because project 4f6830d2e75a47c8a757d4a66c6695ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:36:32.560103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:32.561544 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1986931f-f57c-4f7e-ac20-59cdc9028259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:32.563067 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1301/2599] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:36:32 2022] DELETE /identity/v3/projects/4f6830d2e75a47c8a757d4a66c6695ca => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:32.568286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ac7e3-5481-4b7d-a9d2-af770e23142e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:32.568571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ac7e3-5481-4b7d-a9d2-af770e23142e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:32.568924 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ac7e3-5481-4b7d-a9d2-af770e23142e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:32.596120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-272ac7e3-5481-4b7d-a9d2-af770e23142e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:36:32.597615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-272ac7e3-5481-4b7d-a9d2-af770e23142e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:32.664804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1299/2600] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:36:32.672982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e882f1-f45f-45cd-8102-8ee814435f2f None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:32.673307 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e882f1-f45f-45cd-8102-8ee814435f2f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:32.673580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e882f1-f45f-45cd-8102-8ee814435f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:32.710854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c3e882f1-f45f-45cd-8102-8ee814435f2f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:36:32.710854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c3e882f1-f45f-45cd-8102-8ee814435f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:32.711217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213c89b-cdc1-4c98-ab90-556d5ee8bb68 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:32.711217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213c89b-cdc1-4c98-ab90-556d5ee8bb68 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:32.711217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213c89b-cdc1-4c98-ab90-556d5ee8bb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:32.711813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1302/2601] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:36:32.714094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e882f1-f45f-45cd-8102-8ee814435f2f None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:32.724975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e882f1-f45f-45cd-8102-8ee814435f2f None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:32.729897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4316b02-3020-4648-b1c7-476234b51c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:32.730600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4316b02-3020-4648-b1c7-476234b51c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:32.730974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4316b02-3020-4648-b1c7-476234b51c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:32.731425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4316b02-3020-4648-b1c7-476234b51c4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:32.732063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-2213c89b-cdc1-4c98-ab90-556d5ee8bb68 None None] MFA Rules not processed for user `81b72e82b4a34ab6af29ef440d19f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:36:32.732422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4316b02-3020-4648-b1c7-476234b51c4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:36:32.733174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4316b02-3020-4648-b1c7-476234b51c4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:32.733462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2213c89b-cdc1-4c98-ab90-556d5ee8bb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:32.735226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4316b02-3020-4648-b1c7-476234b51c4b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:32.750058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1303/2602] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:36:32.754787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4316b02-3020-4648-b1c7-476234b51c4b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:32.766167 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4316b02-3020-4648-b1c7-476234b51c4b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:32.771283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239ff48-8a89-4cdb-a8ec-fa33bafdf508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:32.771942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239ff48-8a89-4cdb-a8ec-fa33bafdf508 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:32.772221 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239ff48-8a89-4cdb-a8ec-fa33bafdf508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:32.772525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239ff48-8a89-4cdb-a8ec-fa33bafdf508 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:32.773203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239ff48-8a89-4cdb-a8ec-fa33bafdf508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:32.775160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239ff48-8a89-4cdb-a8ec-fa33bafdf508 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:32.800649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1300/2603] 10.222.0.62 () {58 vars in 967 bytes} [Mon Sep 5 12:36:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:36:32.825058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213c89b-cdc1-4c98-ab90-556d5ee8bb68 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:32.825058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2213c89b-cdc1-4c98-ab90-556d5ee8bb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:32.846649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:32.864174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1304/2604] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:36:32 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:36:32.868743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239ff48-8a89-4cdb-a8ec-fa33bafdf508 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:32.909504 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213c89b-cdc1-4c98-ab90-556d5ee8bb68 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:32.938450 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239ff48-8a89-4cdb-a8ec-fa33bafdf508 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:32.943317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bca6a0-0498-427d-97e6-9adac67a1a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:32.943946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bca6a0-0498-427d-97e6-9adac67a1a36 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:32.944223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bca6a0-0498-427d-97e6-9adac67a1a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:32.944493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bca6a0-0498-427d-97e6-9adac67a1a36 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:32.945127 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bca6a0-0498-427d-97e6-9adac67a1a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:32.947199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bca6a0-0498-427d-97e6-9adac67a1a36 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:32.973267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96341537-e105-4efd-9e9c-f88f89eb08eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b48aa41ff66b42329926bc42379d9800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:32.973877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96341537-e105-4efd-9e9c-f88f89eb08eb service swift] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:32.974153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96341537-e105-4efd-9e9c-f88f89eb08eb service swift] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:32.974422 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96341537-e105-4efd-9e9c-f88f89eb08eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:33.036890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96341537-e105-4efd-9e9c-f88f89eb08eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:33.038548 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96341537-e105-4efd-9e9c-f88f89eb08eb service swift] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:33.047573 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1301/2605] 10.222.0.62 () {60 vars in 1384 bytes} [Mon Sep 5 12:36:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 05 12:36:33.064624 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1305/2606] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:36:32 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:36:33.067652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96341537-e105-4efd-9e9c-f88f89eb08eb service swift] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:33.078669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96341537-e105-4efd-9e9c-f88f89eb08eb service swift] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:33.083297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e751e91-cbc5-436e-b9ac-225c59d61328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:33.083939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e751e91-cbc5-436e-b9ac-225c59d61328 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:33.084216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e751e91-cbc5-436e-b9ac-225c59d61328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:33.084487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e751e91-cbc5-436e-b9ac-225c59d61328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:33.085159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e751e91-cbc5-436e-b9ac-225c59d61328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:33.086735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e751e91-cbc5-436e-b9ac-225c59d61328 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:33.097277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1302/2607] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:36:33.101263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bca6a0-0498-427d-97e6-9adac67a1a36 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:33.112333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bca6a0-0498-427d-97e6-9adac67a1a36 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:33.117602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a771bee-8c3d-40b6-ac61-3a2b1bd64140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:33.118270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a771bee-8c3d-40b6-ac61-3a2b1bd64140 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:33.118561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a771bee-8c3d-40b6-ac61-3a2b1bd64140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:33.118813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a771bee-8c3d-40b6-ac61-3a2b1bd64140 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:33.119463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a771bee-8c3d-40b6-ac61-3a2b1bd64140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:33.121090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a771bee-8c3d-40b6-ac61-3a2b1bd64140 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:33.132639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1306/2608] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:36:33.136856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e751e91-cbc5-436e-b9ac-225c59d61328 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:33.147206 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e751e91-cbc5-436e-b9ac-225c59d61328 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:33.152044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058a72d-3fa8-4e4e-8850-80d33f52dc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:33.152709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058a72d-3fa8-4e4e-8850-80d33f52dc9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:33.152991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058a72d-3fa8-4e4e-8850-80d33f52dc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:33.153267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058a72d-3fa8-4e4e-8850-80d33f52dc9e admin admin] PATH_INFO: `/v3/projects/10fb1dc9d11e45aca91ed128a37889af/users/ceb7fe4ba4644469975c29a74f550b7d/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:33.168735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058a72d-3fa8-4e4e-8850-80d33f52dc9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=10fb1dc9d11e45aca91ed128a37889af, user_id=ceb7fe4ba4644469975c29a74f550b7d, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:33.172014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058a72d-3fa8-4e4e-8850-80d33f52dc9e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:33.228575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1303/2609] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:36:33 2022] PUT /identity/v3/projects/10fb1dc9d11e45aca91ed128a37889af/users/ceb7fe4ba4644469975c29a74f550b7d/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:33.239566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f336dd3-ba85-4ed4-95fc-7a5db4ef4968 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:33.239877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f336dd3-ba85-4ed4-95fc-7a5db4ef4968 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:33.240155 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f336dd3-ba85-4ed4-95fc-7a5db4ef4968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:33.393570 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1f336dd3-ba85-4ed4-95fc-7a5db4ef4968 None None] MFA Rules not processed for user `ceb7fe4ba4644469975c29a74f550b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:36:33.394836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1f336dd3-ba85-4ed4-95fc-7a5db4ef4968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:33.464615 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1307/2610] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:36:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:36:33.469384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058a72d-3fa8-4e4e-8850-80d33f52dc9e admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:33.504387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:33.540768 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058a72d-3fa8-4e4e-8850-80d33f52dc9e admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:33.546184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2ac16-f0c9-4e73-a9ff-16f4a8c58272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:33.546795 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2ac16-f0c9-4e73-a9ff-16f4a8c58272 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:33.547075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2ac16-f0c9-4e73-a9ff-16f4a8c58272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:33.547350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2ac16-f0c9-4e73-a9ff-16f4a8c58272 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:33.548016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2ac16-f0c9-4e73-a9ff-16f4a8c58272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:33.549971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2ac16-f0c9-4e73-a9ff-16f4a8c58272 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:33.709298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1304/2611] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:36:33 2022] POST /identity/v3/projects => generated 373 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:36:33.713230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f336dd3-ba85-4ed4-95fc-7a5db4ef4968 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:33.724507 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f336dd3-ba85-4ed4-95fc-7a5db4ef4968 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:33.732712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:33.795726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927c999-274b-49bf-8fc5-70d7b6796859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:33.796225 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927c999-274b-49bf-8fc5-70d7b6796859 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:33.796508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927c999-274b-49bf-8fc5-70d7b6796859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:33.796808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927c999-274b-49bf-8fc5-70d7b6796859 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:33.797435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927c999-274b-49bf-8fc5-70d7b6796859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:33.799831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927c999-274b-49bf-8fc5-70d7b6796859 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:33.869367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1308/2612] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:36:33 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:36:33.873155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2ac16-f0c9-4e73-a9ff-16f4a8c58272 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:33.884440 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2ac16-f0c9-4e73-a9ff-16f4a8c58272 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:33.889307 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38bc1f-2260-4f18-b921-cadcf4162968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:33.890767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38bc1f-2260-4f18-b921-cadcf4162968 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:33.890767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38bc1f-2260-4f18-b921-cadcf4162968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:33.890968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38bc1f-2260-4f18-b921-cadcf4162968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:33.891587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38bc1f-2260-4f18-b921-cadcf4162968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:33.893221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38bc1f-2260-4f18-b921-cadcf4162968 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:33.904519 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1305/2613] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:36:33.909675 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927c999-274b-49bf-8fc5-70d7b6796859 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:33.920509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927c999-274b-49bf-8fc5-70d7b6796859 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:33.925400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5b433-1d3c-4c78-94ba-7f2947bd2756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:33.926064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5b433-1d3c-4c78-94ba-7f2947bd2756 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:33.926338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5b433-1d3c-4c78-94ba-7f2947bd2756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:33.926612 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5b433-1d3c-4c78-94ba-7f2947bd2756 admin admin] PATH_INFO: `/v3/projects/cec7a661259c40eba76aa173b561043f/users/c78401fd113e4d4886f3cc35afaa9dac/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:33.942211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5b433-1d3c-4c78-94ba-7f2947bd2756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec7a661259c40eba76aa173b561043f, user_id=c78401fd113e4d4886f3cc35afaa9dac, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:33.944923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5b433-1d3c-4c78-94ba-7f2947bd2756 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:34.034729 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1309/2614] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:36:33 2022] PUT /identity/v3/projects/cec7a661259c40eba76aa173b561043f/users/c78401fd113e4d4886f3cc35afaa9dac/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:34.038739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38bc1f-2260-4f18-b921-cadcf4162968 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:34.055457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38bc1f-2260-4f18-b921-cadcf4162968 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:34.122372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ab1e3-38b9-4bef-a18f-36847b627e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:34.122885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ab1e3-38b9-4bef-a18f-36847b627e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:34.123171 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ab1e3-38b9-4bef-a18f-36847b627e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:34.123445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ab1e3-38b9-4bef-a18f-36847b627e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:34.124204 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ab1e3-38b9-4bef-a18f-36847b627e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:34.129607 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ab1e3-38b9-4bef-a18f-36847b627e88 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:34.137341 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1306/2615] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:36:34.141885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5b433-1d3c-4c78-94ba-7f2947bd2756 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:34.152780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5b433-1d3c-4c78-94ba-7f2947bd2756 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:34.157963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485aadd-8f35-403f-8f0e-0342524a3225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:34.159239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485aadd-8f35-403f-8f0e-0342524a3225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:34.159239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485aadd-8f35-403f-8f0e-0342524a3225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:34.159478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485aadd-8f35-403f-8f0e-0342524a3225 admin admin] PATH_INFO: `/v3/projects/cec7a661259c40eba76aa173b561043f/users/c78401fd113e4d4886f3cc35afaa9dac/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:34.161219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485aadd-8f35-403f-8f0e-0342524a3225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec7a661259c40eba76aa173b561043f, user_id=c78401fd113e4d4886f3cc35afaa9dac, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:34.163959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485aadd-8f35-403f-8f0e-0342524a3225 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:34.174691 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1310/2616] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:36:34 2022] PUT /identity/v3/projects/cec7a661259c40eba76aa173b561043f/users/c78401fd113e4d4886f3cc35afaa9dac/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:34.188547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e69f7-e33c-46c4-9d80-e09b07277be6 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:34.188968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e69f7-e33c-46c4-9d80-e09b07277be6 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:34.189251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e69f7-e33c-46c4-9d80-e09b07277be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:34.223965 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7a0e69f7-e33c-46c4-9d80-e09b07277be6 None None] MFA Rules not processed for user `c78401fd113e4d4886f3cc35afaa9dac`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:36:34.225240 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7a0e69f7-e33c-46c4-9d80-e09b07277be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:34.423560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1307/2617] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:36:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:36:34.435048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485aadd-8f35-403f-8f0e-0342524a3225 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:34.436845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6485aadd-8f35-403f-8f0e-0342524a3225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:34.498944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485aadd-8f35-403f-8f0e-0342524a3225 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:34.501869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c603a-6ed1-4401-bf8c-1f410bf483d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:34.502479 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c603a-6ed1-4401-bf8c-1f410bf483d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:34.502751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c603a-6ed1-4401-bf8c-1f410bf483d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:34.503019 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c603a-6ed1-4401-bf8c-1f410bf483d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:34.512666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c603a-6ed1-4401-bf8c-1f410bf483d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:34.514253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c603a-6ed1-4401-bf8c-1f410bf483d5 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:34.525063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1311/2618] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:36:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:36:34.711312 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e69f7-e33c-46c4-9d80-e09b07277be6 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:34.722247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e69f7-e33c-46c4-9d80-e09b07277be6 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:34.725897 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c2ecf-e96c-41a1-9670-54561281d194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:34.726506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c2ecf-e96c-41a1-9670-54561281d194 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:34.726778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c2ecf-e96c-41a1-9670-54561281d194 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:34.727040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c2ecf-e96c-41a1-9670-54561281d194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:34.749048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:34.790414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c2ecf-e96c-41a1-9670-54561281d194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:34.791902 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c2ecf-e96c-41a1-9670-54561281d194 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:34.802736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1308/2619] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:36:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:36:42.364906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c603a-6ed1-4401-bf8c-1f410bf483d5 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:42.366743 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6e6c603a-6ed1-4401-bf8c-1f410bf483d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:42.436147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:42.441254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c603a-6ed1-4401-bf8c-1f410bf483d5 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:42.446386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04645ad7-d2cf-4057-9c47-2b0cc9ac643d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:42.447008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04645ad7-d2cf-4057-9c47-2b0cc9ac643d service placement] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:42.447286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04645ad7-d2cf-4057-9c47-2b0cc9ac643d service placement] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:42.447580 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04645ad7-d2cf-4057-9c47-2b0cc9ac643d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:42.449339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-04645ad7-d2cf-4057-9c47-2b0cc9ac643d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:42.462181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04645ad7-d2cf-4057-9c47-2b0cc9ac643d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:42.463966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04645ad7-d2cf-4057-9c47-2b0cc9ac643d service placement] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:42.480376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1312/2620] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:36:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 12:36:42.553495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c2ecf-e96c-41a1-9670-54561281d194 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:42.555052 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a38c2ecf-e96c-41a1-9670-54561281d194 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:42.631382 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c2ecf-e96c-41a1-9670-54561281d194 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:42.636071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:42.636713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:42.636988 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:42.637259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] PATH_INFO: `/v3/users/087ceea13e3747f49096b42d9d5c1c86` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:42.638412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=087ceea13e3747f49096b42d9d5c1c86)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:42.640416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:42.821571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '087ceea13e3747f49096b42d9d5c1c86'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:42.822224 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '087ceea13e3747f49096b42d9d5c1c86'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:42.902308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '087ceea13e3747f49096b42d9d5c1c86'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:42.916890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '087ceea13e3747f49096b42d9d5c1c86'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:42.917674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '087ceea13e3747f49096b42d9d5c1c86'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:42.930690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1309/2621] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:36:42 2022] DELETE /identity/v3/users/087ceea13e3747f49096b42d9d5c1c86 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:43.319340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04645ad7-d2cf-4057-9c47-2b0cc9ac643d service placement] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:43.331150 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04645ad7-d2cf-4057-9c47-2b0cc9ac643d service placement] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:43.407277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:43.407980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:43.408263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:43.408539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] PATH_INFO: `/v3/projects/6b86bf1c78d74ceaaff6ead90a70526f` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:43.409717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b86bf1c78d74ceaaff6ead90a70526f)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:43.411937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:43.578593 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b86bf1c78d74ceaaff6ead90a70526f', 'request_id': 'req-50fccaa2-7988-42e7-8d12-54b092182f46'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:43.581566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b86bf1c78d74ceaaff6ead90a70526f', 'request_id': 'req-50fccaa2-7988-42e7-8d12-54b092182f46'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:43.666997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b86bf1c78d74ceaaff6ead90a70526f', 'request_id': 'req-50fccaa2-7988-42e7-8d12-54b092182f46'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:43.691210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] The token cache is being invalidate because project 6b86bf1c78d74ceaaff6ead90a70526f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:36:43.691485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:43.692198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:43.693703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1313/2622] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:36:43 2022] DELETE /identity/v3/projects/6b86bf1c78d74ceaaff6ead90a70526f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:43.700633 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:43.714525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:43.720307 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e931d3-6b86-4727-8615-305c49e00348 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:43.768506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:43.789731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:43.790371 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:43.790653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:43.790931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] PATH_INFO: `/v3/users/3470f45a9737400499300a2f3b34ebc7` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:43.792185 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3470f45a9737400499300a2f3b34ebc7)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:43.794103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:44.048969 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3470f45a9737400499300a2f3b34ebc7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:44.049799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3470f45a9737400499300a2f3b34ebc7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:44.126657 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3470f45a9737400499300a2f3b34ebc7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:44.144826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3470f45a9737400499300a2f3b34ebc7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:44.145651 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3470f45a9737400499300a2f3b34ebc7'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:44.163094 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1310/2623] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:36:43 2022] DELETE /identity/v3/users/3470f45a9737400499300a2f3b34ebc7 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:44.476658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:44.478333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:44.492616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fccaa2-7988-42e7-8d12-54b092182f46 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:44.561569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:44.562218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:44.563058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:44.563328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] PATH_INFO: `/v3/projects/e37408e041064b5fac70c14c7bcb80e2` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:44.564662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e37408e041064b5fac70c14c7bcb80e2)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:44.566767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:44.596947 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:44.671530 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e37408e041064b5fac70c14c7bcb80e2', 'request_id': 'req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:44.678587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e37408e041064b5fac70c14c7bcb80e2', 'request_id': 'req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:44.798983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e37408e041064b5fac70c14c7bcb80e2', 'request_id': 'req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:44.812221 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] The token cache is being invalidate because project e37408e041064b5fac70c14c7bcb80e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:36:44.812651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:44.813749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:44.815419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1314/2624] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:36:44 2022] DELETE /identity/v3/projects/e37408e041064b5fac70c14c7bcb80e2 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:49.362769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:49.364229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:49.422480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce178348-5835-4197-84a7-882e691e909a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:49.471926 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c0863-3c89-47f9-8aef-4d238807f857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:49.472534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c0863-3c89-47f9-8aef-4d238807f857 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:49.472807 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c0863-3c89-47f9-8aef-4d238807f857 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:49.473074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c0863-3c89-47f9-8aef-4d238807f857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:49.474823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ed8c0863-3c89-47f9-8aef-4d238807f857 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:49.505481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:49.538687 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c0863-3c89-47f9-8aef-4d238807f857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:49.540155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c0863-3c89-47f9-8aef-4d238807f857 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:49.550330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1311/2625] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:36:53.265222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:53.267444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:53.282064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8d961-bf3e-44d5-8e26-b74f0d79a60d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:53.286837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:53.287473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:53.288610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:53.288610 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] PATH_INFO: `/v3/users/9a61a18900c44bea89647fc9ccdaf110` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:53.289514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a61a18900c44bea89647fc9ccdaf110)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:53.291627 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:53.465229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a61a18900c44bea89647fc9ccdaf110'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:53.466021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a61a18900c44bea89647fc9ccdaf110'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:53.564644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a61a18900c44bea89647fc9ccdaf110'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:53.583466 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a61a18900c44bea89647fc9ccdaf110'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:53.584169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-bfe1623e-69ba-4939-901a-c8882917a592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a61a18900c44bea89647fc9ccdaf110'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:53.597997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1315/2626] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:36:53 2022] DELETE /identity/v3/users/9a61a18900c44bea89647fc9ccdaf110 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:53.939477 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c0863-3c89-47f9-8aef-4d238807f857 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:53.950038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c0863-3c89-47f9-8aef-4d238807f857 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:54.028591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:54.028591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:54.028591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:54.028877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] PATH_INFO: `/v3/projects/094bae935a1e4738b0c51d33c6d89e3a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:54.030034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=094bae935a1e4738b0c51d33c6d89e3a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:54.032127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:54.248802 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094bae935a1e4738b0c51d33c6d89e3a', 'request_id': 'req-54d1ab8f-1d52-4861-853a-a36d78b03534'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:54.255255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094bae935a1e4738b0c51d33c6d89e3a', 'request_id': 'req-54d1ab8f-1d52-4861-853a-a36d78b03534'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:54.354481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094bae935a1e4738b0c51d33c6d89e3a', 'request_id': 'req-54d1ab8f-1d52-4861-853a-a36d78b03534'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:54.366448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] The token cache is being invalidate because project 094bae935a1e4738b0c51d33c6d89e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:36:54.366735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:54.367475 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-54d1ab8f-1d52-4861-853a-a36d78b03534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:36:54.369352 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1312/2627] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:36:53 2022] DELETE /identity/v3/projects/094bae935a1e4738b0c51d33c6d89e3a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:54.374293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92699f2b-be03-4c6d-af79-166637e201dd None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:54.374761 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92699f2b-be03-4c6d-af79-166637e201dd None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:54.374855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92699f2b-be03-4c6d-af79-166637e201dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:54.395199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-92699f2b-be03-4c6d-af79-166637e201dd None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:36:54.396445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-92699f2b-be03-4c6d-af79-166637e201dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:54.455591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1316/2628] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:36:54.463757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278d954-7637-4c4e-b188-c7709771c6e8 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:54.464032 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278d954-7637-4c4e-b188-c7709771c6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:54.464300 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278d954-7637-4c4e-b188-c7709771c6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:54.483825 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f278d954-7637-4c4e-b188-c7709771c6e8 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:36:54.485069 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f278d954-7637-4c4e-b188-c7709771c6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:54.492850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1313/2629] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:36:54.497344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92699f2b-be03-4c6d-af79-166637e201dd None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:54.509064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92699f2b-be03-4c6d-af79-166637e201dd None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:54.513680 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96664a0-ac61-4078-83dc-07dfdf49a19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:54.514272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96664a0-ac61-4078-83dc-07dfdf49a19b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:54.514548 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96664a0-ac61-4078-83dc-07dfdf49a19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:54.514814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96664a0-ac61-4078-83dc-07dfdf49a19b admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:54.515478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96664a0-ac61-4078-83dc-07dfdf49a19b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:36:54.516414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96664a0-ac61-4078-83dc-07dfdf49a19b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:54.518099 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96664a0-ac61-4078-83dc-07dfdf49a19b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:54.530318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1317/2630] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:36:54.534716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278d954-7637-4c4e-b188-c7709771c6e8 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:54.543876 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278d954-7637-4c4e-b188-c7709771c6e8 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:54.548840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a62e1-c3bb-4ef6-83cc-6d973304cc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:54.549436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a62e1-c3bb-4ef6-83cc-6d973304cc43 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:54.549706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a62e1-c3bb-4ef6-83cc-6d973304cc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:54.550001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a62e1-c3bb-4ef6-83cc-6d973304cc43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:54.550617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a62e1-c3bb-4ef6-83cc-6d973304cc43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:54.552530 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a62e1-c3bb-4ef6-83cc-6d973304cc43 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:54.656831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1314/2631] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:36:54 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:36:54.661044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96664a0-ac61-4078-83dc-07dfdf49a19b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:54.671025 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96664a0-ac61-4078-83dc-07dfdf49a19b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:54.678641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:54.727456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9ad71-ef41-40ee-bb9a-94e95347f00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:54.728111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9ad71-ef41-40ee-bb9a-94e95347f00d admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:54.728392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9ad71-ef41-40ee-bb9a-94e95347f00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:54.728666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9ad71-ef41-40ee-bb9a-94e95347f00d admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:54.729292 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9ad71-ef41-40ee-bb9a-94e95347f00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:54.731227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9ad71-ef41-40ee-bb9a-94e95347f00d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:54.828705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1318/2632] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:36:54 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:36:54.832655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a62e1-c3bb-4ef6-83cc-6d973304cc43 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:54.843150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a62e1-c3bb-4ef6-83cc-6d973304cc43 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:54.848672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2dec8-bba3-48dc-a0d5-8da2f2ea8d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:54.849267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2dec8-bba3-48dc-a0d5-8da2f2ea8d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:54.849542 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2dec8-bba3-48dc-a0d5-8da2f2ea8d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:54.849813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2dec8-bba3-48dc-a0d5-8da2f2ea8d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:54.854454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2dec8-bba3-48dc-a0d5-8da2f2ea8d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:54.854454 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2dec8-bba3-48dc-a0d5-8da2f2ea8d91 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:54.862308 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1315/2633] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:36:54.866265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9ad71-ef41-40ee-bb9a-94e95347f00d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:54.876851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9ad71-ef41-40ee-bb9a-94e95347f00d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:54.881694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21013f-982f-4d24-a0c7-393970a24f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:54.882282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21013f-982f-4d24-a0c7-393970a24f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:54.882561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21013f-982f-4d24-a0c7-393970a24f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:54.882831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21013f-982f-4d24-a0c7-393970a24f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:54.883465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21013f-982f-4d24-a0c7-393970a24f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:54.885129 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21013f-982f-4d24-a0c7-393970a24f5b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:54.896142 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1319/2634] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:36:54.900443 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2dec8-bba3-48dc-a0d5-8da2f2ea8d91 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:54.911336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2dec8-bba3-48dc-a0d5-8da2f2ea8d91 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:54.916005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ed11f-4cc7-4a35-b864-d9e0e456dc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:54.916676 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ed11f-4cc7-4a35-b864-d9e0e456dc1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:54.916958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ed11f-4cc7-4a35-b864-d9e0e456dc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:54.917233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ed11f-4cc7-4a35-b864-d9e0e456dc1d admin admin] PATH_INFO: `/v3/projects/7b514c04a13742b1b19d46fec1c40f69/users/8bfb013575d540c59121ac8a1613fc0f/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:54.929733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:54.932953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ed11f-4cc7-4a35-b864-d9e0e456dc1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b514c04a13742b1b19d46fec1c40f69, user_id=8bfb013575d540c59121ac8a1613fc0f, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:54.935735 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ed11f-4cc7-4a35-b864-d9e0e456dc1d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:55.044954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1316/2635] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:36:54 2022] PUT /identity/v3/projects/7b514c04a13742b1b19d46fec1c40f69/users/8bfb013575d540c59121ac8a1613fc0f/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:36:55.051102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbab673-ba67-4de4-91c2-89c657aeb6c1 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:55.051385 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbab673-ba67-4de4-91c2-89c657aeb6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:55.051693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbab673-ba67-4de4-91c2-89c657aeb6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:55.076102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ebbab673-ba67-4de4-91c2-89c657aeb6c1 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:36:55.077338 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ebbab673-ba67-4de4-91c2-89c657aeb6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:55.140183 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1320/2636] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:36:55.150935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ed11f-4cc7-4a35-b864-d9e0e456dc1d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:55.152671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ff8ed11f-4cc7-4a35-b864-d9e0e456dc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:55.209979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ed11f-4cc7-4a35-b864-d9e0e456dc1d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:55.213624 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b92f6-9e17-4d8f-a094-c96ea6b0ec7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:55.214223 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b92f6-9e17-4d8f-a094-c96ea6b0ec7c service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:55.214495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b92f6-9e17-4d8f-a094-c96ea6b0ec7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:55.214762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b92f6-9e17-4d8f-a094-c96ea6b0ec7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:55.227199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b92f6-9e17-4d8f-a094-c96ea6b0ec7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:55.228668 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b92f6-9e17-4d8f-a094-c96ea6b0ec7c service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:55.239146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1317/2637] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:36:58.331463 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbab673-ba67-4de4-91c2-89c657aeb6c1 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:36:58.333057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ebbab673-ba67-4de4-91c2-89c657aeb6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:58.399814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbab673-ba67-4de4-91c2-89c657aeb6c1 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:36:58.405310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97d196-89e6-4870-a42f-0f3c1944b277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:36:58.405310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97d196-89e6-4870-a42f-0f3c1944b277 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:36:58.405310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97d196-89e6-4870-a42f-0f3c1944b277 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:36:58.405310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97d196-89e6-4870-a42f-0f3c1944b277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:36:58.406814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-dc97d196-89e6-4870-a42f-0f3c1944b277 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:36:58.456545 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:36:58.472572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97d196-89e6-4870-a42f-0f3c1944b277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:36:58.474026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97d196-89e6-4870-a42f-0f3c1944b277 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:36:58.483475 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1321/2638] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:36:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:01.237170 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad91cba-1865-44f7-827a-3f262b60605f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:01.237788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad91cba-1865-44f7-827a-3f262b60605f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:01.237788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad91cba-1865-44f7-827a-3f262b60605f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:01.269318 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-bad91cba-1865-44f7-827a-3f262b60605f None None] MFA Rules not processed for user `8bfb013575d540c59121ac8a1613fc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:01.270579 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-bad91cba-1865-44f7-827a-3f262b60605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:01.350532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1318/2639] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:01.367146 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97d196-89e6-4870-a42f-0f3c1944b277 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:01.368506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-dc97d196-89e6-4870-a42f-0f3c1944b277 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:01.470035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97d196-89e6-4870-a42f-0f3c1944b277 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:01.476477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f4bb2-e216-4194-9a21-a97e03669095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:01.477133 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f4bb2-e216-4194-9a21-a97e03669095 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:01.477416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f4bb2-e216-4194-9a21-a97e03669095 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:01.477692 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f4bb2-e216-4194-9a21-a97e03669095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:01.491694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f4bb2-e216-4194-9a21-a97e03669095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:01.491694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f4bb2-e216-4194-9a21-a97e03669095 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:01.507383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1322/2640] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:37:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:04.031165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98f810-8773-4556-b6a2-9970d3d78f70 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:04.031854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98f810-8773-4556-b6a2-9970d3d78f70 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:04.031854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98f810-8773-4556-b6a2-9970d3d78f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:04.053346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4d98f810-8773-4556-b6a2-9970d3d78f70 None None] MFA Rules not processed for user `8bfb013575d540c59121ac8a1613fc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:04.054684 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4d98f810-8773-4556-b6a2-9970d3d78f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:04.069202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1319/2641] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:04.418693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f4bb2-e216-4194-9a21-a97e03669095 service nova] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:04.430136 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f4bb2-e216-4194-9a21-a97e03669095 service nova] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:04.433867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58a4ae-2210-448b-8d36-19e7c36e35a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:04.434481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58a4ae-2210-448b-8d36-19e7c36e35a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:04.434759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58a4ae-2210-448b-8d36-19e7c36e35a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:04.435027 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58a4ae-2210-448b-8d36-19e7c36e35a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:04.436891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ae58a4ae-2210-448b-8d36-19e7c36e35a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:04.447871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58a4ae-2210-448b-8d36-19e7c36e35a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:04.449253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58a4ae-2210-448b-8d36-19e7c36e35a5 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:04.458822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1323/2642] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:37:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:11.390534 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8a1b2-012d-40ff-94f1-475cc70e7069 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:11.391752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8a1b2-012d-40ff-94f1-475cc70e7069 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:11.391986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8a1b2-012d-40ff-94f1-475cc70e7069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:11.411081 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4ca8a1b2-012d-40ff-94f1-475cc70e7069 None None] MFA Rules not processed for user `e7a29212dfae4a93beea7d11085cde63`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:11.412328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4ca8a1b2-012d-40ff-94f1-475cc70e7069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:11.419205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1320/2643] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:14.454618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=47 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:21.405600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:28.941616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58a4ae-2210-448b-8d36-19e7c36e35a5 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:28.943516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ae58a4ae-2210-448b-8d36-19e7c36e35a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:29.048868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58a4ae-2210-448b-8d36-19e7c36e35a5 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:29.060500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:29.061365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:29.061737 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:29.062169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] PATH_INFO: `/v3/users/ceb7fe4ba4644469975c29a74f550b7d` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:29.063728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb7fe4ba4644469975c29a74f550b7d)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:29.065897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:29.305436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb7fe4ba4644469975c29a74f550b7d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:29.306526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb7fe4ba4644469975c29a74f550b7d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:29.383828 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb7fe4ba4644469975c29a74f550b7d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:29.397914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb7fe4ba4644469975c29a74f550b7d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:29.398657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb7fe4ba4644469975c29a74f550b7d'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:29.412561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1324/2644] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:37:28 2022] DELETE /identity/v3/users/ceb7fe4ba4644469975c29a74f550b7d => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:29.417327 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1d789-4ddf-4102-8947-5b52ac526817 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:29.417893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1d789-4ddf-4102-8947-5b52ac526817 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:29.417893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1d789-4ddf-4102-8947-5b52ac526817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:29.455744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6db1d789-4ddf-4102-8947-5b52ac526817 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:29.458643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6db1d789-4ddf-4102-8947-5b52ac526817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:29.535639 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1321/2645] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:29.546126 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:29.550319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:29.644504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685d890-228a-4795-af54-acf426c649f5 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:29.648440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646b1f4-a019-4824-a6dc-147407b90cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:29.649040 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646b1f4-a019-4824-a6dc-147407b90cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:29.649672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646b1f4-a019-4824-a6dc-147407b90cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:29.649672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646b1f4-a019-4824-a6dc-147407b90cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:29.659023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646b1f4-a019-4824-a6dc-147407b90cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:29.660486 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646b1f4-a019-4824-a6dc-147407b90cf4 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:29.664593 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:29.670909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1325/2646] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:30.213269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1d789-4ddf-4102-8947-5b52ac526817 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:30.214719 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6db1d789-4ddf-4102-8947-5b52ac526817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:30.226506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1d789-4ddf-4102-8947-5b52ac526817 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:30.231907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:30.242245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:30.242245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:30.242245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] PATH_INFO: `/v3/projects/10fb1dc9d11e45aca91ed128a37889af` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:30.242245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10fb1dc9d11e45aca91ed128a37889af)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:30.242245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:30.387779 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10fb1dc9d11e45aca91ed128a37889af', 'request_id': 'req-2ee515bc-e26c-4a18-8bea-db4d71271da5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:30.394686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10fb1dc9d11e45aca91ed128a37889af', 'request_id': 'req-2ee515bc-e26c-4a18-8bea-db4d71271da5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:30.504160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10fb1dc9d11e45aca91ed128a37889af', 'request_id': 'req-2ee515bc-e26c-4a18-8bea-db4d71271da5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:30.517600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] The token cache is being invalidate because project 10fb1dc9d11e45aca91ed128a37889af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:37:30.517896 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:30.518582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:30.520115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1322/2647] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:37:30 2022] DELETE /identity/v3/projects/10fb1dc9d11e45aca91ed128a37889af => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:30.524180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646b1f4-a019-4824-a6dc-147407b90cf4 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:30.534053 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646b1f4-a019-4824-a6dc-147407b90cf4 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:30.593701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:30.594375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:30.594652 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:30.594922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] PATH_INFO: `/v3/users/c78401fd113e4d4886f3cc35afaa9dac` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:30.596055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c78401fd113e4d4886f3cc35afaa9dac)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:30.597965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:30.826982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c78401fd113e4d4886f3cc35afaa9dac'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:30.827673 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c78401fd113e4d4886f3cc35afaa9dac'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:30.924540 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c78401fd113e4d4886f3cc35afaa9dac'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:30.938789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c78401fd113e4d4886f3cc35afaa9dac'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:30.939485 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1cf72665-f34d-4efb-bc77-4fa3c431896d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c78401fd113e4d4886f3cc35afaa9dac'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:30.952918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1326/2648] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:37:30 2022] DELETE /identity/v3/users/c78401fd113e4d4886f3cc35afaa9dac => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:31.443258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:31.445263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:31.470762 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee515bc-e26c-4a18-8bea-db4d71271da5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:31.543841 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:31.544551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:31.544788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:31.545130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] PATH_INFO: `/v3/projects/cec7a661259c40eba76aa173b561043f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:31.546695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec7a661259c40eba76aa173b561043f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:31.549288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:31.580001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:31.691840 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec7a661259c40eba76aa173b561043f', 'request_id': 'req-80478527-8b81-4794-af84-0903dbea52e5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:31.697310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec7a661259c40eba76aa173b561043f', 'request_id': 'req-80478527-8b81-4794-af84-0903dbea52e5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:31.796609 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec7a661259c40eba76aa173b561043f', 'request_id': 'req-80478527-8b81-4794-af84-0903dbea52e5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:31.808636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] The token cache is being invalidate because project cec7a661259c40eba76aa173b561043f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:37:31.808954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:31.809618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-80478527-8b81-4794-af84-0903dbea52e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:31.811147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1323/2649] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:37:31 2022] DELETE /identity/v3/projects/cec7a661259c40eba76aa173b561043f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:31.816859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8efe7a-dcb6-4805-9fbb-558c6980f3b8 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:31.817328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8efe7a-dcb6-4805-9fbb-558c6980f3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:31.817448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8efe7a-dcb6-4805-9fbb-558c6980f3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:31.887288 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-0c8efe7a-dcb6-4805-9fbb-558c6980f3b8 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:31.889239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0c8efe7a-dcb6-4805-9fbb-558c6980f3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:31.945633 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:31.958062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1327/2650] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:31.967674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449de71-04fa-46b3-bfce-4301831ab81e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:31.967954 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449de71-04fa-46b3-bfce-4301831ab81e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:31.968224 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449de71-04fa-46b3-bfce-4301831ab81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:31.989766 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6449de71-04fa-46b3-bfce-4301831ab81e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:31.990960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6449de71-04fa-46b3-bfce-4301831ab81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:31.999208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1324/2651] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:32.003923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8efe7a-dcb6-4805-9fbb-558c6980f3b8 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:32.013757 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8efe7a-dcb6-4805-9fbb-558c6980f3b8 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:32.018961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bbe8c-ec45-45ba-9f5c-4a3544a81d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:32.019668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bbe8c-ec45-45ba-9f5c-4a3544a81d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:32.019905 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bbe8c-ec45-45ba-9f5c-4a3544a81d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:32.020163 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bbe8c-ec45-45ba-9f5c-4a3544a81d7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:32.020814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bbe8c-ec45-45ba-9f5c-4a3544a81d7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:37:32.021114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bbe8c-ec45-45ba-9f5c-4a3544a81d7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:32.022722 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bbe8c-ec45-45ba-9f5c-4a3544a81d7d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:32.035915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1328/2652] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:37:32.040557 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449de71-04fa-46b3-bfce-4301831ab81e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:32.050288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449de71-04fa-46b3-bfce-4301831ab81e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:32.055995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6e832-98be-4a19-947c-c64a54545715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:32.056616 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6e832-98be-4a19-947c-c64a54545715 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:32.056887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6e832-98be-4a19-947c-c64a54545715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:32.057153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6e832-98be-4a19-947c-c64a54545715 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:32.057798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6e832-98be-4a19-947c-c64a54545715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:32.059754 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6e832-98be-4a19-947c-c64a54545715 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:32.169667 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1325/2653] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:37:32 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:37:32.173915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bbe8c-ec45-45ba-9f5c-4a3544a81d7d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:32.185904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bbe8c-ec45-45ba-9f5c-4a3544a81d7d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:32.246785 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf923-3192-45b7-af6d-a01b3ba55e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:32.247422 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf923-3192-45b7-af6d-a01b3ba55e6a admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:32.247773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf923-3192-45b7-af6d-a01b3ba55e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:32.248046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf923-3192-45b7-af6d-a01b3ba55e6a admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:32.248686 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccf923-3192-45b7-af6d-a01b3ba55e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:32.251006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccf923-3192-45b7-af6d-a01b3ba55e6a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:32.364887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1329/2654] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:37:32 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:37:32.369115 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6e832-98be-4a19-947c-c64a54545715 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:32.386340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6e832-98be-4a19-947c-c64a54545715 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:32.391036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe0c40-98e7-4c37-95a1-0e1cd1a13610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:32.392238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe0c40-98e7-4c37-95a1-0e1cd1a13610 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:32.392525 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe0c40-98e7-4c37-95a1-0e1cd1a13610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:32.392797 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe0c40-98e7-4c37-95a1-0e1cd1a13610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:32.393466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe0c40-98e7-4c37-95a1-0e1cd1a13610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:32.395087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe0c40-98e7-4c37-95a1-0e1cd1a13610 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:32.407187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1326/2655] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:37:32.409350 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf923-3192-45b7-af6d-a01b3ba55e6a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:32.419505 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf923-3192-45b7-af6d-a01b3ba55e6a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:32.424282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6082b4-07d2-4c1d-84cb-ec11d91d74b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:32.424869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6082b4-07d2-4c1d-84cb-ec11d91d74b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:32.425162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6082b4-07d2-4c1d-84cb-ec11d91d74b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:32.425415 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6082b4-07d2-4c1d-84cb-ec11d91d74b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:32.426048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6082b4-07d2-4c1d-84cb-ec11d91d74b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:32.427662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6082b4-07d2-4c1d-84cb-ec11d91d74b6 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:32.437401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1330/2656] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:37:32.441959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe0c40-98e7-4c37-95a1-0e1cd1a13610 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:32.451720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe0c40-98e7-4c37-95a1-0e1cd1a13610 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:32.457131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25f51f-4ec2-4ed1-bf53-e56ee6502dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:32.457773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25f51f-4ec2-4ed1-bf53-e56ee6502dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:32.458051 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25f51f-4ec2-4ed1-bf53-e56ee6502dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:32.458320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25f51f-4ec2-4ed1-bf53-e56ee6502dc3 admin admin] PATH_INFO: `/v3/projects/8aa4688377de4c08b95d54becc0e8867/users/b7372531b9c54dcab87f7d55c7433c08/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:32.472666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25f51f-4ec2-4ed1-bf53-e56ee6502dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aa4688377de4c08b95d54becc0e8867, user_id=b7372531b9c54dcab87f7d55c7433c08, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:32.475746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25f51f-4ec2-4ed1-bf53-e56ee6502dc3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:32.549690 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1327/2657] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:37:32 2022] PUT /identity/v3/projects/8aa4688377de4c08b95d54becc0e8867/users/b7372531b9c54dcab87f7d55c7433c08/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:32.559854 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11b32f-ce69-4b25-95b1-fa9dcf16be1c None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:32.560186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11b32f-ce69-4b25-95b1-fa9dcf16be1c None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:32.560467 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11b32f-ce69-4b25-95b1-fa9dcf16be1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:32.589391 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8b11b32f-ce69-4b25-95b1-fa9dcf16be1c None None] MFA Rules not processed for user `b7372531b9c54dcab87f7d55c7433c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:32.590622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8b11b32f-ce69-4b25-95b1-fa9dcf16be1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:32.653206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1331/2658] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:32.657678 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25f51f-4ec2-4ed1-bf53-e56ee6502dc3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:32.724694 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25f51f-4ec2-4ed1-bf53-e56ee6502dc3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:32.729782 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9f0f3-07b4-4343-b6dc-7b373dc25f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:32.730386 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f0f3-07b4-4343-b6dc-7b373dc25f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:32.730665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f0f3-07b4-4343-b6dc-7b373dc25f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:32.730953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f0f3-07b4-4343-b6dc-7b373dc25f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:32.731622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9f0f3-07b4-4343-b6dc-7b373dc25f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:32.733575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9f0f3-07b4-4343-b6dc-7b373dc25f39 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:32.837671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1328/2659] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:37:32 2022] POST /identity/v3/projects => generated 369 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:37:32.841734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11b32f-ce69-4b25-95b1-fa9dcf16be1c None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:32.853608 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11b32f-ce69-4b25-95b1-fa9dcf16be1c None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:32.901751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:32.914500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c3e69-6262-486b-b08d-b151802e4d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:32.915117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c3e69-6262-486b-b08d-b151802e4d49 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:32.915401 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c3e69-6262-486b-b08d-b151802e4d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:32.915707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c3e69-6262-486b-b08d-b151802e4d49 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:32.916355 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c3e69-6262-486b-b08d-b151802e4d49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:32.918286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c3e69-6262-486b-b08d-b151802e4d49 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:33.057306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1332/2660] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:37:32 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:37:33.066977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9f0f3-07b4-4343-b6dc-7b373dc25f39 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:33.079304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9f0f3-07b4-4343-b6dc-7b373dc25f39 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:33.084444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c37dd-0bc8-4300-9768-ea70b68a094f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:33.085188 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c37dd-0bc8-4300-9768-ea70b68a094f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:33.085462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c37dd-0bc8-4300-9768-ea70b68a094f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:33.085731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c37dd-0bc8-4300-9768-ea70b68a094f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:33.086413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c37dd-0bc8-4300-9768-ea70b68a094f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:33.088217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c37dd-0bc8-4300-9768-ea70b68a094f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:33.100515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1329/2661] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:37:33.104903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c3e69-6262-486b-b08d-b151802e4d49 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:33.115277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c3e69-6262-486b-b08d-b151802e4d49 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:33.126916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d0ebb-8b34-45c7-9aef-6c54e2b27ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:33.127614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d0ebb-8b34-45c7-9aef-6c54e2b27ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:33.127896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d0ebb-8b34-45c7-9aef-6c54e2b27ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:33.128196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d0ebb-8b34-45c7-9aef-6c54e2b27ca8 admin admin] PATH_INFO: `/v3/projects/325758b180004bd9ba2d77fd5d980a8b/users/216075007fbd4bd4883c65d454dab02e/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:33.151376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d0ebb-8b34-45c7-9aef-6c54e2b27ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=325758b180004bd9ba2d77fd5d980a8b, user_id=216075007fbd4bd4883c65d454dab02e, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:33.154046 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d0ebb-8b34-45c7-9aef-6c54e2b27ca8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:33.213088 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1333/2662] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:37:33 2022] PUT /identity/v3/projects/325758b180004bd9ba2d77fd5d980a8b/users/216075007fbd4bd4883c65d454dab02e/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:33.217109 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c37dd-0bc8-4300-9768-ea70b68a094f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:33.227199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c37dd-0bc8-4300-9768-ea70b68a094f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:33.239957 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:33.288426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ce0c3-f9dc-47ac-a18a-ef3050985b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:33.288974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ce0c3-f9dc-47ac-a18a-ef3050985b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:33.289257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ce0c3-f9dc-47ac-a18a-ef3050985b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:33.289528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ce0c3-f9dc-47ac-a18a-ef3050985b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:33.290209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ce0c3-f9dc-47ac-a18a-ef3050985b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:33.291855 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ce0c3-f9dc-47ac-a18a-ef3050985b63 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:33.302213 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1330/2663] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:37:33.306532 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d0ebb-8b34-45c7-9aef-6c54e2b27ca8 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:33.318952 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d0ebb-8b34-45c7-9aef-6c54e2b27ca8 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:33.332283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd60e53-6b80-4557-b3a9-d1b7e48361a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:33.332283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd60e53-6b80-4557-b3a9-d1b7e48361a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:33.332283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd60e53-6b80-4557-b3a9-d1b7e48361a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:33.332283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd60e53-6b80-4557-b3a9-d1b7e48361a1 admin admin] PATH_INFO: `/v3/projects/325758b180004bd9ba2d77fd5d980a8b/users/216075007fbd4bd4883c65d454dab02e/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:33.332754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd60e53-6b80-4557-b3a9-d1b7e48361a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=325758b180004bd9ba2d77fd5d980a8b, user_id=216075007fbd4bd4883c65d454dab02e, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:33.332754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd60e53-6b80-4557-b3a9-d1b7e48361a1 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:33.344237 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1334/2664] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:37:33 2022] PUT /identity/v3/projects/325758b180004bd9ba2d77fd5d980a8b/users/216075007fbd4bd4883c65d454dab02e/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:33.352281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72805154-fdcf-43ac-bfe9-dbfd62fc7f88 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:33.352812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72805154-fdcf-43ac-bfe9-dbfd62fc7f88 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:33.353160 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72805154-fdcf-43ac-bfe9-dbfd62fc7f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:33.383033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-72805154-fdcf-43ac-bfe9-dbfd62fc7f88 None None] MFA Rules not processed for user `216075007fbd4bd4883c65d454dab02e`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:33.384400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-72805154-fdcf-43ac-bfe9-dbfd62fc7f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:33.466043 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1331/2665] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:33.477602 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd60e53-6b80-4557-b3a9-d1b7e48361a1 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:33.479412 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4dd60e53-6b80-4557-b3a9-d1b7e48361a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:33.538295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd60e53-6b80-4557-b3a9-d1b7e48361a1 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:33.542124 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311f39e-9f2e-4b95-9d3a-93ea6056c33c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:33.543353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311f39e-9f2e-4b95-9d3a-93ea6056c33c service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:33.543353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311f39e-9f2e-4b95-9d3a-93ea6056c33c service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:33.543353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311f39e-9f2e-4b95-9d3a-93ea6056c33c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:33.553679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311f39e-9f2e-4b95-9d3a-93ea6056c33c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:33.555118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311f39e-9f2e-4b95-9d3a-93ea6056c33c service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:33.565992 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1335/2666] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:37:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:34.841086 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72805154-fdcf-43ac-bfe9-dbfd62fc7f88 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:34.842611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-72805154-fdcf-43ac-bfe9-dbfd62fc7f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:34.929771 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72805154-fdcf-43ac-bfe9-dbfd62fc7f88 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:34.933568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671baa1-9100-4001-b4eb-8976060ca528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:34.934205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671baa1-9100-4001-b4eb-8976060ca528 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:34.934487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671baa1-9100-4001-b4eb-8976060ca528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:34.934757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671baa1-9100-4001-b4eb-8976060ca528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:34.936587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c671baa1-9100-4001-b4eb-8976060ca528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:34.972649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=100 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:35.007030 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671baa1-9100-4001-b4eb-8976060ca528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:35.009361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671baa1-9100-4001-b4eb-8976060ca528 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:35.025679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1332/2667] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:37:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:36.831516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311f39e-9f2e-4b95-9d3a-93ea6056c33c service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:36.899765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311f39e-9f2e-4b95-9d3a-93ea6056c33c service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:36.904823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:36.905475 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:36.905774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:36.906044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] PATH_INFO: `/v3/users/b7372531b9c54dcab87f7d55c7433c08` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:36.907209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7372531b9c54dcab87f7d55c7433c08)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:36.909768 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:37.140809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:37.142836 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7372531b9c54dcab87f7d55c7433c08'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:37.143623 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7372531b9c54dcab87f7d55c7433c08'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:37.270561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7372531b9c54dcab87f7d55c7433c08'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:37.285059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7372531b9c54dcab87f7d55c7433c08'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:37.285848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7372531b9c54dcab87f7d55c7433c08'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:37.301178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1336/2668] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:37:36 2022] DELETE /identity/v3/users/b7372531b9c54dcab87f7d55c7433c08 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:37.307528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b2668-3b37-441b-b770-c8e35147dade None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:37.308418 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b2668-3b37-441b-b770-c8e35147dade None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:37.308696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b2668-3b37-441b-b770-c8e35147dade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:37.329101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b06b2668-3b37-441b-b770-c8e35147dade None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:37.330326 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b06b2668-3b37-441b-b770-c8e35147dade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:37.400722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1333/2669] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:37.412498 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:37.414621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:37.504159 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ef899-6752-4825-88c2-3b0eaa5bc173 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:37.508730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8eae59-0d91-4cf3-b6fc-c191d64ee6cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:37.509379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eae59-0d91-4cf3-b6fc-c191d64ee6cb service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:37.509657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eae59-0d91-4cf3-b6fc-c191d64ee6cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:37.509925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eae59-0d91-4cf3-b6fc-c191d64ee6cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:37.521772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8eae59-0d91-4cf3-b6fc-c191d64ee6cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:37.525976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8eae59-0d91-4cf3-b6fc-c191d64ee6cb service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:37.533431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1337/2670] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:38.414640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b2668-3b37-441b-b770-c8e35147dade None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:38.416228 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b06b2668-3b37-441b-b770-c8e35147dade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:38.434551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b2668-3b37-441b-b770-c8e35147dade None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:38.444058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:38.445124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:38.445674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:38.446161 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] PATH_INFO: `/v3/projects/8aa4688377de4c08b95d54becc0e8867` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:38.447736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aa4688377de4c08b95d54becc0e8867)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:38.450058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:38.564168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aa4688377de4c08b95d54becc0e8867', 'request_id': 'req-31d56b3f-f4aa-41a8-a771-64890fcd58ba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:38.569845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aa4688377de4c08b95d54becc0e8867', 'request_id': 'req-31d56b3f-f4aa-41a8-a771-64890fcd58ba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:38.656302 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aa4688377de4c08b95d54becc0e8867', 'request_id': 'req-31d56b3f-f4aa-41a8-a771-64890fcd58ba'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:38.668912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] The token cache is being invalidate because project 8aa4688377de4c08b95d54becc0e8867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:37:38.669194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:38.669848 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:38.671317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1334/2671] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:37:38 2022] DELETE /identity/v3/projects/8aa4688377de4c08b95d54becc0e8867 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:38.676159 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8eae59-0d91-4cf3-b6fc-c191d64ee6cb service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:38.686051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8eae59-0d91-4cf3-b6fc-c191d64ee6cb service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:38.750831 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:38.751469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:38.751792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:38.752068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] PATH_INFO: `/v3/users/216075007fbd4bd4883c65d454dab02e` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:38.753191 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216075007fbd4bd4883c65d454dab02e)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:38.755211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:38.975622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216075007fbd4bd4883c65d454dab02e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:38.976200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216075007fbd4bd4883c65d454dab02e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:39.093332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216075007fbd4bd4883c65d454dab02e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:39.104734 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:39.106708 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216075007fbd4bd4883c65d454dab02e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:39.107468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-85bf1602-72c8-4baa-ab65-36b3c1e5fac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216075007fbd4bd4883c65d454dab02e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:39.119668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1338/2672] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:37:38 2022] DELETE /identity/v3/users/216075007fbd4bd4883c65d454dab02e => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:39.622376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:39.626289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:39.639275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d56b3f-f4aa-41a8-a771-64890fcd58ba admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:39.692441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:39.701436 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:39.702078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:39.702353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:39.702629 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] PATH_INFO: `/v3/projects/325758b180004bd9ba2d77fd5d980a8b` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:39.703819 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] RBAC: Authorizing `identity:delete_project(project_id=325758b180004bd9ba2d77fd5d980a8b)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:39.705758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:39.858987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '325758b180004bd9ba2d77fd5d980a8b', 'request_id': 'req-659392a7-e8e4-4e05-be32-9e12be30116f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:39.865187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '325758b180004bd9ba2d77fd5d980a8b', 'request_id': 'req-659392a7-e8e4-4e05-be32-9e12be30116f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:39.995466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '325758b180004bd9ba2d77fd5d980a8b', 'request_id': 'req-659392a7-e8e4-4e05-be32-9e12be30116f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:40.008656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] The token cache is being invalidate because project 325758b180004bd9ba2d77fd5d980a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:37:40.008949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:40.009601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-659392a7-e8e4-4e05-be32-9e12be30116f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:37:40.011085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1335/2673] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:37:39 2022] DELETE /identity/v3/projects/325758b180004bd9ba2d77fd5d980a8b => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:40.016174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e39608-afa2-4d1d-a405-2e9bf105e00e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:40.016689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e39608-afa2-4d1d-a405-2e9bf105e00e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:40.016826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e39608-afa2-4d1d-a405-2e9bf105e00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:40.036789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c0e39608-afa2-4d1d-a405-2e9bf105e00e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:40.037984 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c0e39608-afa2-4d1d-a405-2e9bf105e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:40.100424 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1339/2674] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:40.108262 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dae556-01f2-4206-b6e7-d7fb962cc9b0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:40.108552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dae556-01f2-4206-b6e7-d7fb962cc9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:40.108831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dae556-01f2-4206-b6e7-d7fb962cc9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:40.130131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-40dae556-01f2-4206-b6e7-d7fb962cc9b0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:40.131257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-40dae556-01f2-4206-b6e7-d7fb962cc9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:40.139257 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1336/2675] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:40.143554 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e39608-afa2-4d1d-a405-2e9bf105e00e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:40.153776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e39608-afa2-4d1d-a405-2e9bf105e00e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:40.158443 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38857bb-66c7-4e90-96d4-70052cbf6c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:40.159164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38857bb-66c7-4e90-96d4-70052cbf6c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:40.159451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38857bb-66c7-4e90-96d4-70052cbf6c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:40.159758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38857bb-66c7-4e90-96d4-70052cbf6c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:40.160468 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38857bb-66c7-4e90-96d4-70052cbf6c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:37:40.160770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38857bb-66c7-4e90-96d4-70052cbf6c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:40.162430 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38857bb-66c7-4e90-96d4-70052cbf6c06 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:40.174703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1340/2676] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:37:40.179786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dae556-01f2-4206-b6e7-d7fb962cc9b0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:40.189084 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dae556-01f2-4206-b6e7-d7fb962cc9b0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:40.194442 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df893821-42d6-462f-a614-657809128fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:40.195048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df893821-42d6-462f-a614-657809128fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:40.195328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df893821-42d6-462f-a614-657809128fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:40.195641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df893821-42d6-462f-a614-657809128fc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:40.196272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df893821-42d6-462f-a614-657809128fc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:40.198153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df893821-42d6-462f-a614-657809128fc5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:40.288361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1337/2677] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:37:40 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:37:40.292500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38857bb-66c7-4e90-96d4-70052cbf6c06 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:40.303497 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38857bb-66c7-4e90-96d4-70052cbf6c06 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:40.361704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26d154-10ca-492d-baba-b088b3703054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:40.362316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26d154-10ca-492d-baba-b088b3703054 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:40.362620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26d154-10ca-492d-baba-b088b3703054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:40.362896 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26d154-10ca-492d-baba-b088b3703054 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:40.363598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26d154-10ca-492d-baba-b088b3703054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:40.365521 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26d154-10ca-492d-baba-b088b3703054 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:40.453470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1341/2678] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:37:40 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:37:40.462423 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df893821-42d6-462f-a614-657809128fc5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:40.473399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df893821-42d6-462f-a614-657809128fc5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:40.478057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8934ec-495e-44d9-9b0b-1b628429b84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:40.478649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8934ec-495e-44d9-9b0b-1b628429b84b admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:40.478922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8934ec-495e-44d9-9b0b-1b628429b84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:40.479187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8934ec-495e-44d9-9b0b-1b628429b84b admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:40.479898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8934ec-495e-44d9-9b0b-1b628429b84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:40.481459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8934ec-495e-44d9-9b0b-1b628429b84b admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:40.491298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1338/2679] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:37:40.495280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26d154-10ca-492d-baba-b088b3703054 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:40.505782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26d154-10ca-492d-baba-b088b3703054 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:40.510813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16957e0-9d0f-4832-beac-c7c6b58f2df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:40.511419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16957e0-9d0f-4832-beac-c7c6b58f2df7 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:40.511739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16957e0-9d0f-4832-beac-c7c6b58f2df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:40.512015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16957e0-9d0f-4832-beac-c7c6b58f2df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:40.512657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16957e0-9d0f-4832-beac-c7c6b58f2df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:40.514262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16957e0-9d0f-4832-beac-c7c6b58f2df7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:40.525804 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1342/2680] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:37:40.530069 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8934ec-495e-44d9-9b0b-1b628429b84b admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:40.540746 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8934ec-495e-44d9-9b0b-1b628429b84b admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:40.545500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3122ee-901d-4a4a-9050-18b615e6ef6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:40.546137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3122ee-901d-4a4a-9050-18b615e6ef6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:40.546411 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3122ee-901d-4a4a-9050-18b615e6ef6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:40.546681 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3122ee-901d-4a4a-9050-18b615e6ef6a admin admin] PATH_INFO: `/v3/projects/1e7b7f62953f4b7c9a46727007d45e26/users/f562a04546684404b582e9f89b3c970b/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:40.561381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3122ee-901d-4a4a-9050-18b615e6ef6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e7b7f62953f4b7c9a46727007d45e26, user_id=f562a04546684404b582e9f89b3c970b, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:40.564140 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3122ee-901d-4a4a-9050-18b615e6ef6a admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:40.670636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1339/2681] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:37:40 2022] PUT /identity/v3/projects/1e7b7f62953f4b7c9a46727007d45e26/users/f562a04546684404b582e9f89b3c970b/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:40.675816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e4f21-6ff0-4017-8a5b-21b4bdda91b9 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:40.676101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e4f21-6ff0-4017-8a5b-21b4bdda91b9 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:40.676377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e4f21-6ff0-4017-8a5b-21b4bdda91b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:40.697174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-729e4f21-6ff0-4017-8a5b-21b4bdda91b9 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:40.698383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-729e4f21-6ff0-4017-8a5b-21b4bdda91b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:40.748775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:40.764715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1343/2682] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:40.775470 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3122ee-901d-4a4a-9050-18b615e6ef6a admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:40.777209 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6d3122ee-901d-4a4a-9050-18b615e6ef6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:40.834251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3122ee-901d-4a4a-9050-18b615e6ef6a admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:40.837945 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a611a-6bae-47fc-af3d-07161ad40c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:40.838549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a611a-6bae-47fc-af3d-07161ad40c4b service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:40.838827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a611a-6bae-47fc-af3d-07161ad40c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:40.839099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a611a-6bae-47fc-af3d-07161ad40c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:40.848951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a611a-6bae-47fc-af3d-07161ad40c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:40.850371 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a611a-6bae-47fc-af3d-07161ad40c4b service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:40.859890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1340/2683] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:42.269344 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41684a82-87c1-458c-9426-940c8b7d7131 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:42.269955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41684a82-87c1-458c-9426-940c8b7d7131 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:42.269955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41684a82-87c1-458c-9426-940c8b7d7131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:42.298383 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-41684a82-87c1-458c-9426-940c8b7d7131 None None] MFA Rules not processed for user `f562a04546684404b582e9f89b3c970b`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:42.299878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-41684a82-87c1-458c-9426-940c8b7d7131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:42.369742 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1344/2684] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:42.374449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a611a-6bae-47fc-af3d-07161ad40c4b service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:42.383967 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a611a-6bae-47fc-af3d-07161ad40c4b service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:42.388453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc95d5d-5559-4b38-8365-ade63a8f5339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:42.389221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc95d5d-5559-4b38-8365-ade63a8f5339 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:42.389358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc95d5d-5559-4b38-8365-ade63a8f5339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:42.389612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc95d5d-5559-4b38-8365-ade63a8f5339 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:42.390360 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc95d5d-5559-4b38-8365-ade63a8f5339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:42.392264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc95d5d-5559-4b38-8365-ade63a8f5339 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:42.471247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:42.478350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1341/2685] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:37:42 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:37:42.482351 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41684a82-87c1-458c-9426-940c8b7d7131 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:42.493204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41684a82-87c1-458c-9426-940c8b7d7131 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:42.550022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782ce4f-7781-4d2f-b1ed-1a1e05c92207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:42.550637 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782ce4f-7781-4d2f-b1ed-1a1e05c92207 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:42.550918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782ce4f-7781-4d2f-b1ed-1a1e05c92207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:42.551190 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782ce4f-7781-4d2f-b1ed-1a1e05c92207 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:42.551848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782ce4f-7781-4d2f-b1ed-1a1e05c92207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:42.553799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782ce4f-7781-4d2f-b1ed-1a1e05c92207 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:42.665436 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1345/2686] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:37:42 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:37:42.669455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc95d5d-5559-4b38-8365-ade63a8f5339 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:42.680042 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc95d5d-5559-4b38-8365-ade63a8f5339 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:42.684711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193ef3b-16dd-43a8-ab18-225c130fbb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:42.685311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193ef3b-16dd-43a8-ab18-225c130fbb99 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:42.685592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193ef3b-16dd-43a8-ab18-225c130fbb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:42.685866 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193ef3b-16dd-43a8-ab18-225c130fbb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:42.686503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193ef3b-16dd-43a8-ab18-225c130fbb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:42.688148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193ef3b-16dd-43a8-ab18-225c130fbb99 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:42.700538 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1342/2687] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:37:42.704630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782ce4f-7781-4d2f-b1ed-1a1e05c92207 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:42.716214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782ce4f-7781-4d2f-b1ed-1a1e05c92207 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:42.720936 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d6da6-24fb-4b46-918d-424934c131a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:42.721582 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d6da6-24fb-4b46-918d-424934c131a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:42.722480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d6da6-24fb-4b46-918d-424934c131a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:42.722781 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d6da6-24fb-4b46-918d-424934c131a9 admin admin] PATH_INFO: `/v3/projects/e536c0f74ccf43cc9c6c8b5194833880/users/fca674a5f96840c891744e4053c0235e/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:42.737516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d6da6-24fb-4b46-918d-424934c131a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e536c0f74ccf43cc9c6c8b5194833880, user_id=fca674a5f96840c891744e4053c0235e, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:42.740196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d6da6-24fb-4b46-918d-424934c131a9 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:42.799315 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1346/2688] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:37:42 2022] PUT /identity/v3/projects/e536c0f74ccf43cc9c6c8b5194833880/users/fca674a5f96840c891744e4053c0235e/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:42.803336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193ef3b-16dd-43a8-ab18-225c130fbb99 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:42.813948 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193ef3b-16dd-43a8-ab18-225c130fbb99 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:42.872342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36611b-ce77-4f1b-b349-f9c7ca63403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:42.872949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36611b-ce77-4f1b-b349-f9c7ca63403c admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:42.873221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36611b-ce77-4f1b-b349-f9c7ca63403c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:42.873489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36611b-ce77-4f1b-b349-f9c7ca63403c admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:42.874127 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36611b-ce77-4f1b-b349-f9c7ca63403c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:42.875726 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36611b-ce77-4f1b-b349-f9c7ca63403c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:42.885938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1343/2689] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:37:42.890166 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d6da6-24fb-4b46-918d-424934c131a9 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:42.901234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d6da6-24fb-4b46-918d-424934c131a9 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:42.906048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7f9f9-e8f7-4173-9221-324a256a45a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:42.906704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7f9f9-e8f7-4173-9221-324a256a45a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:42.906985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7f9f9-e8f7-4173-9221-324a256a45a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:42.907262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7f9f9-e8f7-4173-9221-324a256a45a7 admin admin] PATH_INFO: `/v3/projects/e536c0f74ccf43cc9c6c8b5194833880/users/fca674a5f96840c891744e4053c0235e/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:42.909272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7f9f9-e8f7-4173-9221-324a256a45a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e536c0f74ccf43cc9c6c8b5194833880, user_id=fca674a5f96840c891744e4053c0235e, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:42.912023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7f9f9-e8f7-4173-9221-324a256a45a7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:42.922306 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1347/2690] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:37:42 2022] PUT /identity/v3/projects/e536c0f74ccf43cc9c6c8b5194833880/users/fca674a5f96840c891744e4053c0235e/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:37:44.237276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8ad9-3483-4627-b83c-8dc94f03f133 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:44.238489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8ad9-3483-4627-b83c-8dc94f03f133 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:44.238489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8ad9-3483-4627-b83c-8dc94f03f133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:44.270020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b27e8ad9-3483-4627-b83c-8dc94f03f133 None None] MFA Rules not processed for user `fca674a5f96840c891744e4053c0235e`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:37:44.270377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b27e8ad9-3483-4627-b83c-8dc94f03f133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:44.351047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1344/2691] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:37:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:37:44.367080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7f9f9-e8f7-4173-9221-324a256a45a7 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:44.368864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-cab7f9f9-e8f7-4173-9221-324a256a45a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:44.408100 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:44.449456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7f9f9-e8f7-4173-9221-324a256a45a7 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:44.453303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c824-6c34-4768-b3a0-1c0dc353d858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:44.453907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c824-6c34-4768-b3a0-1c0dc353d858 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:44.454185 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c824-6c34-4768-b3a0-1c0dc353d858 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:44.454455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c824-6c34-4768-b3a0-1c0dc353d858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:44.512187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0c824-6c34-4768-b3a0-1c0dc353d858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:44.513639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0c824-6c34-4768-b3a0-1c0dc353d858 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:44.522941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1348/2692] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:37:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:46.268143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8ad9-3483-4627-b83c-8dc94f03f133 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:37:46.270028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b27e8ad9-3483-4627-b83c-8dc94f03f133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:37:46.281890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8ad9-3483-4627-b83c-8dc94f03f133 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:37:46.286063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b0384-8d0f-4f5d-8806-fda2ea981fea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:37:46.286653 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b0384-8d0f-4f5d-8806-fda2ea981fea service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:37:46.286933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b0384-8d0f-4f5d-8806-fda2ea981fea service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:37:46.287205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b0384-8d0f-4f5d-8806-fda2ea981fea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:37:46.296310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b0384-8d0f-4f5d-8806-fda2ea981fea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:37:46.297730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b0384-8d0f-4f5d-8806-fda2ea981fea service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:37:46.307227 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1345/2693] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:37:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:37:54.536775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=42 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:37:56.303005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=74 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:02.893974 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c824-6c34-4768-b3a0-1c0dc353d858 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:02.957340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c824-6c34-4768-b3a0-1c0dc353d858 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:02.961929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:02.962562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:02.962837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:02.963128 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] PATH_INFO: `/v3/users/f562a04546684404b582e9f89b3c970b` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:02.964283 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f562a04546684404b582e9f89b3c970b)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:02.966208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:03.237762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f562a04546684404b582e9f89b3c970b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:03.238927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f562a04546684404b582e9f89b3c970b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:03.349103 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f562a04546684404b582e9f89b3c970b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:03.362419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f562a04546684404b582e9f89b3c970b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:03.363705 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f562a04546684404b582e9f89b3c970b'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:03.375955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1349/2694] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:02 2022] DELETE /identity/v3/users/f562a04546684404b582e9f89b3c970b => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:03.626911 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b0384-8d0f-4f5d-8806-fda2ea981fea service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:03.640003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b0384-8d0f-4f5d-8806-fda2ea981fea service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:03.698748 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:03.699385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:03.699697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:03.699972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] PATH_INFO: `/v3/projects/1e7b7f62953f4b7c9a46727007d45e26` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:03.701154 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e7b7f62953f4b7c9a46727007d45e26)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:03.702987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:03.818283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e7b7f62953f4b7c9a46727007d45e26', 'request_id': 'req-255a1555-b53f-4255-8a4f-0e7dfaa8c200'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:03.824023 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e7b7f62953f4b7c9a46727007d45e26', 'request_id': 'req-255a1555-b53f-4255-8a4f-0e7dfaa8c200'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:03.898974 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e7b7f62953f4b7c9a46727007d45e26', 'request_id': 'req-255a1555-b53f-4255-8a4f-0e7dfaa8c200'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:03.911106 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] The token cache is being invalidate because project 1e7b7f62953f4b7c9a46727007d45e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:03.911391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:03.912078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:03.913564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1346/2695] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:03 2022] DELETE /identity/v3/projects/1e7b7f62953f4b7c9a46727007d45e26 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:38:03.917557 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:03.922259 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:03.932438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b183b1-ed9e-49fe-aa46-629f9c270769 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:03.991427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:03.992113 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:03.992392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:03.992693 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] PATH_INFO: `/v3/users/fca674a5f96840c891744e4053c0235e` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:03.993868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca674a5f96840c891744e4053c0235e)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:03.995977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:04.028068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:04.265462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca674a5f96840c891744e4053c0235e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.266029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca674a5f96840c891744e4053c0235e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.336779 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca674a5f96840c891744e4053c0235e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.350440 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca674a5f96840c891744e4053c0235e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.351101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8e676ea5-fb7f-45cc-bc91-3522c3c3b64c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca674a5f96840c891744e4053c0235e'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.364727 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1350/2696] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:03 2022] DELETE /identity/v3/users/fca674a5f96840c891744e4053c0235e => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:04.607179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:04.608790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:04.622665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a1555-b53f-4255-8a4f-0e7dfaa8c200 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:04.681136 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:04.682063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:04.682063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:04.682368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] PATH_INFO: `/v3/projects/e536c0f74ccf43cc9c6c8b5194833880` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:04.683429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e536c0f74ccf43cc9c6c8b5194833880)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:04.685367 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:04.783155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:04.796448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e536c0f74ccf43cc9c6c8b5194833880', 'request_id': 'req-249a1718-f48c-4566-9d00-ae68221eb8f3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.801949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e536c0f74ccf43cc9c6c8b5194833880', 'request_id': 'req-249a1718-f48c-4566-9d00-ae68221eb8f3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.888014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e536c0f74ccf43cc9c6c8b5194833880', 'request_id': 'req-249a1718-f48c-4566-9d00-ae68221eb8f3'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.901353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] The token cache is being invalidate because project e536c0f74ccf43cc9c6c8b5194833880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:04.901640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.902298 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-249a1718-f48c-4566-9d00-ae68221eb8f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:04.903829 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1347/2697] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:04 2022] DELETE /identity/v3/projects/e536c0f74ccf43cc9c6c8b5194833880 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:04.909755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441c399-3ef1-4c9f-a63a-f6a73772c81f None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:04.910039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441c399-3ef1-4c9f-a63a-f6a73772c81f None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:04.910316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441c399-3ef1-4c9f-a63a-f6a73772c81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:04.931259 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-7441c399-3ef1-4c9f-a63a-f6a73772c81f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:04.932512 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7441c399-3ef1-4c9f-a63a-f6a73772c81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:04.991765 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1351/2698] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:38:05.000674 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d29fb1-b1cd-47f1-ab09-11e3d3d28e38 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.000970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d29fb1-b1cd-47f1-ab09-11e3d3d28e38 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.001276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d29fb1-b1cd-47f1-ab09-11e3d3d28e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.024561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-41d29fb1-b1cd-47f1-ab09-11e3d3d28e38 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:05.025813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-41d29fb1-b1cd-47f1-ab09-11e3d3d28e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:05.038314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1348/2699] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:05.039317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441c399-3ef1-4c9f-a63a-f6a73772c81f None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.049833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441c399-3ef1-4c9f-a63a-f6a73772c81f None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.055741 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0044d-df32-4818-a994-9218eb29f77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.056366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0044d-df32-4818-a994-9218eb29f77f admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.056643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0044d-df32-4818-a994-9218eb29f77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.056918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0044d-df32-4818-a994-9218eb29f77f admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.057596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0044d-df32-4818-a994-9218eb29f77f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:38:05.058147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0044d-df32-4818-a994-9218eb29f77f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.061621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0044d-df32-4818-a994-9218eb29f77f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.078154 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1352/2700] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:38:05.082515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d29fb1-b1cd-47f1-ab09-11e3d3d28e38 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.092182 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d29fb1-b1cd-47f1-ab09-11e3d3d28e38 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.096772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb401a26-dd01-4153-8a88-4713aea76ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.097369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401a26-dd01-4153-8a88-4713aea76ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.097641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401a26-dd01-4153-8a88-4713aea76ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.097910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401a26-dd01-4153-8a88-4713aea76ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.098531 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb401a26-dd01-4153-8a88-4713aea76ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.100463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb401a26-dd01-4153-8a88-4713aea76ea9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.258939 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1349/2701] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:05 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:38:05.260437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0044d-df32-4818-a994-9218eb29f77f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.271316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0044d-df32-4818-a994-9218eb29f77f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.329456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f952c8-1562-4466-80ff-302d2ff165a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.329979 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f952c8-1562-4466-80ff-302d2ff165a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.330253 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f952c8-1562-4466-80ff-302d2ff165a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.330525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f952c8-1562-4466-80ff-302d2ff165a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.331158 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f952c8-1562-4466-80ff-302d2ff165a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.333331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f952c8-1562-4466-80ff-302d2ff165a7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.453877 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1353/2702] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:05 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:05.458288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb401a26-dd01-4153-8a88-4713aea76ea9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.468585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb401a26-dd01-4153-8a88-4713aea76ea9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.473744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70931aa7-d368-4aa2-91c2-e479d2c6f0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.474347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70931aa7-d368-4aa2-91c2-e479d2c6f0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.474626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70931aa7-d368-4aa2-91c2-e479d2c6f0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.474907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70931aa7-d368-4aa2-91c2-e479d2c6f0f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.475643 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70931aa7-d368-4aa2-91c2-e479d2c6f0f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.477237 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70931aa7-d368-4aa2-91c2-e479d2c6f0f0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.487456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1350/2703] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:05.492492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f952c8-1562-4466-80ff-302d2ff165a7 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.502967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f952c8-1562-4466-80ff-302d2ff165a7 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.508107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8017fe2-c576-4dfe-9f44-4160e1ecaca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.508702 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8017fe2-c576-4dfe-9f44-4160e1ecaca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.508995 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8017fe2-c576-4dfe-9f44-4160e1ecaca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.509266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8017fe2-c576-4dfe-9f44-4160e1ecaca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.509907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8017fe2-c576-4dfe-9f44-4160e1ecaca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.511504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8017fe2-c576-4dfe-9f44-4160e1ecaca1 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.522794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1354/2704] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:05.526998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70931aa7-d368-4aa2-91c2-e479d2c6f0f0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.537078 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70931aa7-d368-4aa2-91c2-e479d2c6f0f0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.542247 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcdcc4-de78-405d-93ea-8d063d3d2ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.545272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcdcc4-de78-405d-93ea-8d063d3d2ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.545272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcdcc4-de78-405d-93ea-8d063d3d2ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.545272 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcdcc4-de78-405d-93ea-8d063d3d2ddb admin admin] PATH_INFO: `/v3/projects/7287d836308c4382939183c4fad4b23c/users/013f9e9c108d4cb3a334c102ee31ac40/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.559899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcdcc4-de78-405d-93ea-8d063d3d2ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7287d836308c4382939183c4fad4b23c, user_id=013f9e9c108d4cb3a334c102ee31ac40, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.562528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcdcc4-de78-405d-93ea-8d063d3d2ddb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.631564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1351/2705] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:05 2022] PUT /identity/v3/projects/7287d836308c4382939183c4fad4b23c/users/013f9e9c108d4cb3a334c102ee31ac40/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:38:05.647653 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3fa83-7839-4246-924f-873caabcfa95 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.647938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3fa83-7839-4246-924f-873caabcfa95 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.648215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3fa83-7839-4246-924f-873caabcfa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.656591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:05.678164 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e7b3fa83-7839-4246-924f-873caabcfa95 None None] MFA Rules not processed for user `013f9e9c108d4cb3a334c102ee31ac40`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:05.679086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e7b3fa83-7839-4246-924f-873caabcfa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:05.744002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1355/2706] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:05.749074 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcdcc4-de78-405d-93ea-8d063d3d2ddb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.817493 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcdcc4-de78-405d-93ea-8d063d3d2ddb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.822439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66934b33-063d-4bab-9aab-0b06951686bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.823040 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66934b33-063d-4bab-9aab-0b06951686bf admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.823313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66934b33-063d-4bab-9aab-0b06951686bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.823611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66934b33-063d-4bab-9aab-0b06951686bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.824220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66934b33-063d-4bab-9aab-0b06951686bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.826102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66934b33-063d-4bab-9aab-0b06951686bf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:05.919784 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1352/2707] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:05 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:05.923975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3fa83-7839-4246-924f-873caabcfa95 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:05.935176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3fa83-7839-4246-924f-873caabcfa95 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:05.993181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb5d1c-2e25-454a-9da6-41b7b3196688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:05.993793 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb5d1c-2e25-454a-9da6-41b7b3196688 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:05.994067 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb5d1c-2e25-454a-9da6-41b7b3196688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:05.994337 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb5d1c-2e25-454a-9da6-41b7b3196688 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:05.994973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb5d1c-2e25-454a-9da6-41b7b3196688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:05.996916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb5d1c-2e25-454a-9da6-41b7b3196688 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.111316 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1356/2708] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:05 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:06.115938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66934b33-063d-4bab-9aab-0b06951686bf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.126821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66934b33-063d-4bab-9aab-0b06951686bf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.132235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1f31b-060b-4bb6-b023-fe4f0645a92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.132856 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1f31b-060b-4bb6-b023-fe4f0645a92f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.133124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1f31b-060b-4bb6-b023-fe4f0645a92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.133389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1f31b-060b-4bb6-b023-fe4f0645a92f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.134036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1f31b-060b-4bb6-b023-fe4f0645a92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.135672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1f31b-060b-4bb6-b023-fe4f0645a92f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.148787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1353/2709] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:38:06.152980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb5d1c-2e25-454a-9da6-41b7b3196688 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.163380 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb5d1c-2e25-454a-9da6-41b7b3196688 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.168108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a732e-2a7f-49b3-954a-f52034a15522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.168770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a732e-2a7f-49b3-954a-f52034a15522 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.169048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a732e-2a7f-49b3-954a-f52034a15522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.169322 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a732e-2a7f-49b3-954a-f52034a15522 admin admin] PATH_INFO: `/v3/projects/4f990a90813e47709c90fdfcf790823f/users/fd70312c4676490f97b622eb0b2751b4/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.184762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a732e-2a7f-49b3-954a-f52034a15522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f990a90813e47709c90fdfcf790823f, user_id=fd70312c4676490f97b622eb0b2751b4, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.187378 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a732e-2a7f-49b3-954a-f52034a15522 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.254574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1357/2710] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:06 2022] PUT /identity/v3/projects/4f990a90813e47709c90fdfcf790823f/users/fd70312c4676490f97b622eb0b2751b4/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:06.258930 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1f31b-060b-4bb6-b023-fe4f0645a92f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.270566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1f31b-060b-4bb6-b023-fe4f0645a92f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.303319 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:06.331277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472b375-bd95-49b8-86de-1e75dac575d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.332466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472b375-bd95-49b8-86de-1e75dac575d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.332806 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472b375-bd95-49b8-86de-1e75dac575d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.333080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472b375-bd95-49b8-86de-1e75dac575d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.333721 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472b375-bd95-49b8-86de-1e75dac575d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.335346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472b375-bd95-49b8-86de-1e75dac575d3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.345522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1354/2711] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:06.350108 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a732e-2a7f-49b3-954a-f52034a15522 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.360969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a732e-2a7f-49b3-954a-f52034a15522 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.365917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70570303-7c7e-440f-8e36-fad21f4ada78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.366579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70570303-7c7e-440f-8e36-fad21f4ada78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.366859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70570303-7c7e-440f-8e36-fad21f4ada78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.367135 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70570303-7c7e-440f-8e36-fad21f4ada78 admin admin] PATH_INFO: `/v3/projects/4f990a90813e47709c90fdfcf790823f/users/fd70312c4676490f97b622eb0b2751b4/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.369119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70570303-7c7e-440f-8e36-fad21f4ada78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f990a90813e47709c90fdfcf790823f, user_id=fd70312c4676490f97b622eb0b2751b4, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.371823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70570303-7c7e-440f-8e36-fad21f4ada78 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.383826 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1358/2712] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:06 2022] PUT /identity/v3/projects/4f990a90813e47709c90fdfcf790823f/users/fd70312c4676490f97b622eb0b2751b4/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:06.394758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414f251-e05d-4da4-a14b-e04fd990cf73 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.395075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414f251-e05d-4da4-a14b-e04fd990cf73 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.395381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414f251-e05d-4da4-a14b-e04fd990cf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.424441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f414f251-e05d-4da4-a14b-e04fd990cf73 None None] MFA Rules not processed for user `fd70312c4676490f97b622eb0b2751b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:06.426193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f414f251-e05d-4da4-a14b-e04fd990cf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:06.495950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1355/2713] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:06.507261 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70570303-7c7e-440f-8e36-fad21f4ada78 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:06.509080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-70570303-7c7e-440f-8e36-fad21f4ada78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:06.561411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:06.566598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70570303-7c7e-440f-8e36-fad21f4ada78 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:06.570480 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9401-e88b-4ac9-a85e-2c4c529a9e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:06.571110 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9401-e88b-4ac9-a85e-2c4c529a9e93 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:06.571389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9401-e88b-4ac9-a85e-2c4c529a9e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:06.571694 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9401-e88b-4ac9-a85e-2c4c529a9e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:06.629696 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb9401-e88b-4ac9-a85e-2c4c529a9e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:06.631841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb9401-e88b-4ac9-a85e-2c4c529a9e93 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:06.649690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1359/2714] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:09.761125 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414f251-e05d-4da4-a14b-e04fd990cf73 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:09.827861 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414f251-e05d-4da4-a14b-e04fd990cf73 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:09.833070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:09.833712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:09.833991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:09.834267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] PATH_INFO: `/v3/users/013f9e9c108d4cb3a334c102ee31ac40` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:09.835435 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=013f9e9c108d4cb3a334c102ee31ac40)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:09.837392 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.041580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '013f9e9c108d4cb3a334c102ee31ac40'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:10.042313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '013f9e9c108d4cb3a334c102ee31ac40'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:10.225376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '013f9e9c108d4cb3a334c102ee31ac40'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:10.226728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:10.237944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '013f9e9c108d4cb3a334c102ee31ac40'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:10.238798 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '013f9e9c108d4cb3a334c102ee31ac40'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:10.251793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1356/2715] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:09 2022] DELETE /identity/v3/users/013f9e9c108d4cb3a334c102ee31ac40 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:10.258681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48efeb-e9f1-4ff8-8e45-3b4f36a11532 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.259209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48efeb-e9f1-4ff8-8e45-3b4f36a11532 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.260617 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48efeb-e9f1-4ff8-8e45-3b4f36a11532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.280631 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8b48efeb-e9f1-4ff8-8e45-3b4f36a11532 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:10.281837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8b48efeb-e9f1-4ff8-8e45-3b4f36a11532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:10.353219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1360/2716] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:10.363724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.365647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:10.434301 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a348e2d-86b4-4896-9b48-7f3ad74d8ba9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.437958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b070d10-a6bc-4b08-9077-f35a9f7a0191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.438561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b070d10-a6bc-4b08-9077-f35a9f7a0191 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.438843 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b070d10-a6bc-4b08-9077-f35a9f7a0191 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.439118 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b070d10-a6bc-4b08-9077-f35a9f7a0191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.450288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b070d10-a6bc-4b08-9077-f35a9f7a0191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.451818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b070d10-a6bc-4b08-9077-f35a9f7a0191 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:10.462437 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1357/2717] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:10.897035 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48efeb-e9f1-4ff8-8e45-3b4f36a11532 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:10.898471 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8b48efeb-e9f1-4ff8-8e45-3b4f36a11532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:10.917586 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48efeb-e9f1-4ff8-8e45-3b4f36a11532 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:10.922664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:10.923328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:10.930303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:10.930303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] PATH_INFO: `/v3/projects/7287d836308c4382939183c4fad4b23c` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:10.930303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7287d836308c4382939183c4fad4b23c)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:10.930303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:11.086636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7287d836308c4382939183c4fad4b23c', 'request_id': 'req-1da38e24-2d5f-45e4-b669-7290803a5767'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:11.093290 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7287d836308c4382939183c4fad4b23c', 'request_id': 'req-1da38e24-2d5f-45e4-b669-7290803a5767'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:11.160070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7287d836308c4382939183c4fad4b23c', 'request_id': 'req-1da38e24-2d5f-45e4-b669-7290803a5767'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:11.172285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] The token cache is being invalidate because project 7287d836308c4382939183c4fad4b23c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:11.172569 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:11.173230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:11.174716 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1361/2718] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:10 2022] DELETE /identity/v3/projects/7287d836308c4382939183c4fad4b23c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:11.179450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b070d10-a6bc-4b08-9077-f35a9f7a0191 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:11.189613 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b070d10-a6bc-4b08-9077-f35a9f7a0191 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:11.248330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:11.248962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:11.249238 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:11.249602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] PATH_INFO: `/v3/users/fd70312c4676490f97b622eb0b2751b4` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:11.250728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd70312c4676490f97b622eb0b2751b4)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:11.252655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:11.501562 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd70312c4676490f97b622eb0b2751b4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:11.502313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd70312c4676490f97b622eb0b2751b4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:11.573964 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd70312c4676490f97b622eb0b2751b4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:11.586938 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd70312c4676490f97b622eb0b2751b4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:11.587686 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-776cb132-6777-45b1-bedc-c23cde86c1dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd70312c4676490f97b622eb0b2751b4'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:11.592903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:11.602079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1358/2719] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:11 2022] DELETE /identity/v3/users/fd70312c4676490f97b622eb0b2751b4 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:12.050604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:12.052302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:12.068818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da38e24-2d5f-45e4-b669-7290803a5767 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:12.093409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:12.131397 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:12.132088 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:12.132448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:12.132915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] PATH_INFO: `/v3/projects/4f990a90813e47709c90fdfcf790823f` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:12.134126 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f990a90813e47709c90fdfcf790823f)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:12.136543 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:12.261457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f990a90813e47709c90fdfcf790823f', 'request_id': 'req-1563c62c-4265-46bf-8b44-59ad8b459e12'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:12.267869 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f990a90813e47709c90fdfcf790823f', 'request_id': 'req-1563c62c-4265-46bf-8b44-59ad8b459e12'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:12.366211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f990a90813e47709c90fdfcf790823f', 'request_id': 'req-1563c62c-4265-46bf-8b44-59ad8b459e12'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:12.378559 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] The token cache is being invalidate because project 4f990a90813e47709c90fdfcf790823f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:12.378880 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:12.379629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-1563c62c-4265-46bf-8b44-59ad8b459e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:12.381043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1362/2720] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:12 2022] DELETE /identity/v3/projects/4f990a90813e47709c90fdfcf790823f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:12.386036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff0c76-2bc3-477d-b486-5ae50794bc49 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:12.386546 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff0c76-2bc3-477d-b486-5ae50794bc49 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:12.386646 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff0c76-2bc3-477d-b486-5ae50794bc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:12.407919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ceff0c76-2bc3-477d-b486-5ae50794bc49 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:12.409144 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ceff0c76-2bc3-477d-b486-5ae50794bc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:12.471749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1359/2721] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:12.480006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d323c-6ef4-498f-9cdf-ed215ccb6d50 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:12.480317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d323c-6ef4-498f-9cdf-ed215ccb6d50 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:12.481695 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d323c-6ef4-498f-9cdf-ed215ccb6d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:12.503953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-ff5d323c-6ef4-498f-9cdf-ed215ccb6d50 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:12.508901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ff5d323c-6ef4-498f-9cdf-ed215ccb6d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:12.513365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1363/2722] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:12.519303 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff0c76-2bc3-477d-b486-5ae50794bc49 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:12.530429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff0c76-2bc3-477d-b486-5ae50794bc49 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:12.535568 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abf994-07fa-4ed5-b1e0-078ba891f124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:12.536731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abf994-07fa-4ed5-b1e0-078ba891f124 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:12.537340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abf994-07fa-4ed5-b1e0-078ba891f124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:12.537340 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abf994-07fa-4ed5-b1e0-078ba891f124 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:12.537903 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abf994-07fa-4ed5-b1e0-078ba891f124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:38:12.538199 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abf994-07fa-4ed5-b1e0-078ba891f124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:12.539835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abf994-07fa-4ed5-b1e0-078ba891f124 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:12.558119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1360/2723] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:38:12.559462 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d323c-6ef4-498f-9cdf-ed215ccb6d50 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:12.569094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d323c-6ef4-498f-9cdf-ed215ccb6d50 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:12.574015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012d929-3847-4a79-88cc-2b9da47c27e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:12.574629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012d929-3847-4a79-88cc-2b9da47c27e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:12.574907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012d929-3847-4a79-88cc-2b9da47c27e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:12.575179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012d929-3847-4a79-88cc-2b9da47c27e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:12.576106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012d929-3847-4a79-88cc-2b9da47c27e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:12.577802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012d929-3847-4a79-88cc-2b9da47c27e7 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:12.684002 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1364/2724] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:12 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:12.688593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abf994-07fa-4ed5-b1e0-078ba891f124 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:12.698989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abf994-07fa-4ed5-b1e0-078ba891f124 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:12.758014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d914db9-4e3d-4aa5-8f1a-74a3fa9e1b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:12.758626 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d914db9-4e3d-4aa5-8f1a-74a3fa9e1b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:12.758906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d914db9-4e3d-4aa5-8f1a-74a3fa9e1b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:12.759184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d914db9-4e3d-4aa5-8f1a-74a3fa9e1b05 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:12.759878 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d914db9-4e3d-4aa5-8f1a-74a3fa9e1b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:12.761799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d914db9-4e3d-4aa5-8f1a-74a3fa9e1b05 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:12.868527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1361/2725] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:12 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:12.869784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012d929-3847-4a79-88cc-2b9da47c27e7 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:12.881144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012d929-3847-4a79-88cc-2b9da47c27e7 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:12.886246 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d3ecc-30d7-4678-a122-10a78a1a5e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:12.886859 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d3ecc-30d7-4678-a122-10a78a1a5e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:12.887134 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d3ecc-30d7-4678-a122-10a78a1a5e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:12.887403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d3ecc-30d7-4678-a122-10a78a1a5e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:12.888085 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d3ecc-30d7-4678-a122-10a78a1a5e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:12.889682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d3ecc-30d7-4678-a122-10a78a1a5e64 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:12.901357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1365/2726] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:38:12.905380 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d914db9-4e3d-4aa5-8f1a-74a3fa9e1b05 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:12.915812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d914db9-4e3d-4aa5-8f1a-74a3fa9e1b05 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:12.920457 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983524ee-b35e-4709-9aaa-a1b9c846f1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:12.921057 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983524ee-b35e-4709-9aaa-a1b9c846f1cb admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:12.921330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983524ee-b35e-4709-9aaa-a1b9c846f1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:12.921706 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983524ee-b35e-4709-9aaa-a1b9c846f1cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:12.922338 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983524ee-b35e-4709-9aaa-a1b9c846f1cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:12.923968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983524ee-b35e-4709-9aaa-a1b9c846f1cb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:12.935877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1362/2727] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:12.940152 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d3ecc-30d7-4678-a122-10a78a1a5e64 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:12.950915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d3ecc-30d7-4678-a122-10a78a1a5e64 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:12.955918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26f31c-677b-4bc2-b275-fca205ad6629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:12.956575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26f31c-677b-4bc2-b275-fca205ad6629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:12.956850 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26f31c-677b-4bc2-b275-fca205ad6629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:12.957124 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26f31c-677b-4bc2-b275-fca205ad6629 admin admin] PATH_INFO: `/v3/projects/6de8b66f1cb747fd8003c8043a71e89c/users/06b340634e8349caa1ed8c3d7e2af288/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:12.972906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26f31c-677b-4bc2-b275-fca205ad6629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de8b66f1cb747fd8003c8043a71e89c, user_id=06b340634e8349caa1ed8c3d7e2af288, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:12.975596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26f31c-677b-4bc2-b275-fca205ad6629 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:13.083671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1366/2728] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:12 2022] PUT /identity/v3/projects/6de8b66f1cb747fd8003c8043a71e89c/users/06b340634e8349caa1ed8c3d7e2af288/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:13.152063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3b90f-5e19-48ae-ae66-a15fedbad874 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:13.152290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3b90f-5e19-48ae-ae66-a15fedbad874 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:13.152531 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3b90f-5e19-48ae-ae66-a15fedbad874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:13.181656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-54a3b90f-5e19-48ae-ae66-a15fedbad874 None None] MFA Rules not processed for user `06b340634e8349caa1ed8c3d7e2af288`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:13.182886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-54a3b90f-5e19-48ae-ae66-a15fedbad874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:13.232103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:13.250539 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1363/2729] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:13.261864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26f31c-677b-4bc2-b275-fca205ad6629 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:13.263587 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ee26f31c-677b-4bc2-b275-fca205ad6629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:13.322669 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26f31c-677b-4bc2-b275-fca205ad6629 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:13.326368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d3d6a-cbc5-4cb4-92b1-353cc16c6415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:13.326975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d3d6a-cbc5-4cb4-92b1-353cc16c6415 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:13.327255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d3d6a-cbc5-4cb4-92b1-353cc16c6415 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:13.327566 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d3d6a-cbc5-4cb4-92b1-353cc16c6415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:13.333042 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:13.338740 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d3d6a-cbc5-4cb4-92b1-353cc16c6415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:13.340296 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d3d6a-cbc5-4cb4-92b1-353cc16c6415 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:13.349648 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1367/2730] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:17.524243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3b90f-5e19-48ae-ae66-a15fedbad874 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:17.587935 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3b90f-5e19-48ae-ae66-a15fedbad874 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:17.592544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:17.593173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:17.593448 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:17.593716 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] PATH_INFO: `/v3/users/06b340634e8349caa1ed8c3d7e2af288` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:17.594835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b340634e8349caa1ed8c3d7e2af288)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:17.596781 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:17.814872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b340634e8349caa1ed8c3d7e2af288'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:17.815693 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b340634e8349caa1ed8c3d7e2af288'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:17.917119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b340634e8349caa1ed8c3d7e2af288'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:17.929744 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b340634e8349caa1ed8c3d7e2af288'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:17.931031 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b340634e8349caa1ed8c3d7e2af288'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:17.943944 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1364/2731] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:38:17 2022] DELETE /identity/v3/users/06b340634e8349caa1ed8c3d7e2af288 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:17.948549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85b593-8124-46b4-a668-3fe37992c1c6 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:17.949023 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85b593-8124-46b4-a668-3fe37992c1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:17.949143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85b593-8124-46b4-a668-3fe37992c1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:17.969841 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-5c85b593-8124-46b4-a668-3fe37992c1c6 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:17.971079 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5c85b593-8124-46b4-a668-3fe37992c1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:18.032173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1368/2732] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:38:18.042536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:18.044073 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:18.108738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fe44f-935c-40bc-b48e-12ee91aba954 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:18.112555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e5935-0eb9-4cfc-bb60-cf046788c7b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:18.113167 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e5935-0eb9-4cfc-bb60-cf046788c7b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:18.113445 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e5935-0eb9-4cfc-bb60-cf046788c7b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:18.113743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e5935-0eb9-4cfc-bb60-cf046788c7b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:18.123260 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e5935-0eb9-4cfc-bb60-cf046788c7b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:18.124725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e5935-0eb9-4cfc-bb60-cf046788c7b6 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:18.135335 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1365/2733] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:18.657922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85b593-8124-46b4-a668-3fe37992c1c6 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:18.660687 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5c85b593-8124-46b4-a668-3fe37992c1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:18.681785 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85b593-8124-46b4-a668-3fe37992c1c6 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:18.686775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:18.687416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:18.687733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:18.688014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] PATH_INFO: `/v3/projects/6de8b66f1cb747fd8003c8043a71e89c` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:18.691266 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de8b66f1cb747fd8003c8043a71e89c)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:18.694010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:18.787088 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de8b66f1cb747fd8003c8043a71e89c', 'request_id': 'req-7e31d2b8-3df8-435d-9110-9c5686415546'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:18.792819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de8b66f1cb747fd8003c8043a71e89c', 'request_id': 'req-7e31d2b8-3df8-435d-9110-9c5686415546'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:18.888411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de8b66f1cb747fd8003c8043a71e89c', 'request_id': 'req-7e31d2b8-3df8-435d-9110-9c5686415546'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:18.900999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] The token cache is being invalidate because project 6de8b66f1cb747fd8003c8043a71e89c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:38:18.901282 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:18.902039 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7e31d2b8-3df8-435d-9110-9c5686415546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:38:18.903609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1369/2734] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:38:18 2022] DELETE /identity/v3/projects/6de8b66f1cb747fd8003c8043a71e89c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:38:18.908614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d27f8b-bd5b-4e52-b71c-1aeb4c06a00f None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:18.909103 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d27f8b-bd5b-4e52-b71c-1aeb4c06a00f None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:18.909189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d27f8b-bd5b-4e52-b71c-1aeb4c06a00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:18.932095 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-b8d27f8b-bd5b-4e52-b71c-1aeb4c06a00f None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:18.933309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-b8d27f8b-bd5b-4e52-b71c-1aeb4c06a00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:18.938880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:18.993088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1366/2735] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:19.001091 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52740119-4dd0-474e-a74b-c1938f381558 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:19.001635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52740119-4dd0-474e-a74b-c1938f381558 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:19.001635 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52740119-4dd0-474e-a74b-c1938f381558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:19.022583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-52740119-4dd0-474e-a74b-c1938f381558 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:19.023861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-52740119-4dd0-474e-a74b-c1938f381558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:19.032192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1370/2736] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:19.037033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d27f8b-bd5b-4e52-b71c-1aeb4c06a00f None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:19.047017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d27f8b-bd5b-4e52-b71c-1aeb4c06a00f None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:19.053317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a53f6-49b2-4f64-b3f4-421b946150e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:19.053996 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a53f6-49b2-4f64-b3f4-421b946150e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:19.058514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a53f6-49b2-4f64-b3f4-421b946150e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:19.058514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a53f6-49b2-4f64-b3f4-421b946150e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:19.058514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a53f6-49b2-4f64-b3f4-421b946150e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:38:19.058514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a53f6-49b2-4f64-b3f4-421b946150e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:19.058514 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a53f6-49b2-4f64-b3f4-421b946150e3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:19.070003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1367/2737] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:38:19.074409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52740119-4dd0-474e-a74b-c1938f381558 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:19.083967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52740119-4dd0-474e-a74b-c1938f381558 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:19.104698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5fe46-403a-4789-98fb-036b720751ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:19.104698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5fe46-403a-4789-98fb-036b720751ee admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:19.104698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5fe46-403a-4789-98fb-036b720751ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:19.104698 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5fe46-403a-4789-98fb-036b720751ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:19.105262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5fe46-403a-4789-98fb-036b720751ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:19.105262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5fe46-403a-4789-98fb-036b720751ee admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:19.410270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1371/2738] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:38:19 2022] POST /identity/v3/projects => generated 383 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:19.415060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a53f6-49b2-4f64-b3f4-421b946150e3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:19.425405 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a53f6-49b2-4f64-b3f4-421b946150e3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:19.487523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5807c-111d-4519-aae8-25f3d78a32e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:19.488156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5807c-111d-4519-aae8-25f3d78a32e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:19.488431 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5807c-111d-4519-aae8-25f3d78a32e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:19.488714 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5807c-111d-4519-aae8-25f3d78a32e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:19.489467 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5807c-111d-4519-aae8-25f3d78a32e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:19.491399 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5807c-111d-4519-aae8-25f3d78a32e3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:19.622602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1368/2739] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:38:19 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:38:19.626833 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5fe46-403a-4789-98fb-036b720751ee admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:19.635343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5fe46-403a-4789-98fb-036b720751ee admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:19.639968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef7d8b-8779-4216-a32d-ade2f8dbae42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:19.640700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef7d8b-8779-4216-a32d-ade2f8dbae42 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:19.640978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef7d8b-8779-4216-a32d-ade2f8dbae42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:19.641249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef7d8b-8779-4216-a32d-ade2f8dbae42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:19.641886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef7d8b-8779-4216-a32d-ade2f8dbae42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:19.643499 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef7d8b-8779-4216-a32d-ade2f8dbae42 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:19.656179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1372/2740] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:19.662947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5807c-111d-4519-aae8-25f3d78a32e3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:19.671229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5807c-111d-4519-aae8-25f3d78a32e3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:19.677652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b96c1-6570-4652-b92a-f94cfb091ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:19.678372 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b96c1-6570-4652-b92a-f94cfb091ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:19.678731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b96c1-6570-4652-b92a-f94cfb091ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:19.680235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b96c1-6570-4652-b92a-f94cfb091ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:19.680235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b96c1-6570-4652-b92a-f94cfb091ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:19.681490 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b96c1-6570-4652-b92a-f94cfb091ff2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:19.693489 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1369/2741] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:38:19.697759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef7d8b-8779-4216-a32d-ade2f8dbae42 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:19.708701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef7d8b-8779-4216-a32d-ade2f8dbae42 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:19.713425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3238b-a121-4e55-8a76-0e313a3a4bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:19.714241 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3238b-a121-4e55-8a76-0e313a3a4bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:19.714583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3238b-a121-4e55-8a76-0e313a3a4bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:19.714921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3238b-a121-4e55-8a76-0e313a3a4bf6 admin admin] PATH_INFO: `/v3/projects/2a77deb0bf54420e90c26e66699b3611/users/6c22d28f87c941afb1fad1cfaf3a5fb2/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:19.730639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3238b-a121-4e55-8a76-0e313a3a4bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a77deb0bf54420e90c26e66699b3611, user_id=6c22d28f87c941afb1fad1cfaf3a5fb2, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:19.733340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3238b-a121-4e55-8a76-0e313a3a4bf6 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:19.808731 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1373/2742] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:38:19 2022] PUT /identity/v3/projects/2a77deb0bf54420e90c26e66699b3611/users/6c22d28f87c941afb1fad1cfaf3a5fb2/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:38:19.813838 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665905ec-5eb6-424e-8edc-b4a91191c7b5 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:19.814179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665905ec-5eb6-424e-8edc-b4a91191c7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:19.814463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665905ec-5eb6-424e-8edc-b4a91191c7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:19.836415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-665905ec-5eb6-424e-8edc-b4a91191c7b5 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:19.837617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-665905ec-5eb6-424e-8edc-b4a91191c7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:19.898536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1370/2743] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:19.908311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3238b-a121-4e55-8a76-0e313a3a4bf6 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:19.910043 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-72e3238b-a121-4e55-8a76-0e313a3a4bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:19.925095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:19.969819 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3238b-a121-4e55-8a76-0e313a3a4bf6 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:19.973574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec07823-e51c-4aa2-8fd2-6ecfa59843bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:19.974190 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec07823-e51c-4aa2-8fd2-6ecfa59843bc service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:19.974472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec07823-e51c-4aa2-8fd2-6ecfa59843bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:19.974748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec07823-e51c-4aa2-8fd2-6ecfa59843bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:19.984081 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec07823-e51c-4aa2-8fd2-6ecfa59843bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:19.985525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec07823-e51c-4aa2-8fd2-6ecfa59843bc service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:19.996061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1374/2744] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:21.274011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0602b6-ad52-4797-8903-eb5fd61fbdab None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:21.274011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0602b6-ad52-4797-8903-eb5fd61fbdab None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:21.274011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0602b6-ad52-4797-8903-eb5fd61fbdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:21.294173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-1e0602b6-ad52-4797-8903-eb5fd61fbdab None None] MFA Rules not processed for user `6c22d28f87c941afb1fad1cfaf3a5fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:21.295450 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1e0602b6-ad52-4797-8903-eb5fd61fbdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:21.301350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:21.371246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1371/2745] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:21.383300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec07823-e51c-4aa2-8fd2-6ecfa59843bc service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:21.385081 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-bec07823-e51c-4aa2-8fd2-6ecfa59843bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:21.443174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec07823-e51c-4aa2-8fd2-6ecfa59843bc service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:21.446888 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a07756-8a48-425d-8c22-10dfb5bfa01f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:21.447495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a07756-8a48-425d-8c22-10dfb5bfa01f service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:21.447824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a07756-8a48-425d-8c22-10dfb5bfa01f service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:21.448179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a07756-8a48-425d-8c22-10dfb5bfa01f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:21.457172 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a07756-8a48-425d-8c22-10dfb5bfa01f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:21.458678 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a07756-8a48-425d-8c22-10dfb5bfa01f service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:21.468173 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1375/2746] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:23.142177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005a5f9-8e54-4402-bdf0-3f2cc688f789 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:23.143047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005a5f9-8e54-4402-bdf0-3f2cc688f789 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:23.143047 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005a5f9-8e54-4402-bdf0-3f2cc688f789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:23.165417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-e005a5f9-8e54-4402-bdf0-3f2cc688f789 None None] MFA Rules not processed for user `6c22d28f87c941afb1fad1cfaf3a5fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:38:23.166677 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-e005a5f9-8e54-4402-bdf0-3f2cc688f789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:23.174190 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1372/2747] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:38:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:38:23.446494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a07756-8a48-425d-8c22-10dfb5bfa01f service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:38:23.448077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-64a07756-8a48-425d-8c22-10dfb5bfa01f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:23.552435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a07756-8a48-425d-8c22-10dfb5bfa01f service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:38:23.558036 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f02dc5-966b-4d00-b0d5-c5a3399d98b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:38:23.558663 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f02dc5-966b-4d00-b0d5-c5a3399d98b8 service placement] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:38:23.558946 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f02dc5-966b-4d00-b0d5-c5a3399d98b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:38:23.559222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f02dc5-966b-4d00-b0d5-c5a3399d98b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:38:23.561008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b6f02dc5-966b-4d00-b0d5-c5a3399d98b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:38:23.575161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f02dc5-966b-4d00-b0d5-c5a3399d98b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:38:23.576714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f02dc5-966b-4d00-b0d5-c5a3399d98b8 service placement] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:38:23.587752 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1376/2748] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:38:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:38:33.159634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=27 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:38:33.583390 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=87 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:14.862593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005a5f9-8e54-4402-bdf0-3f2cc688f789 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:14.872428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005a5f9-8e54-4402-bdf0-3f2cc688f789 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:14.877982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c7bdc-139b-455a-86b2-34c612727d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:14.877982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c7bdc-139b-455a-86b2-34c612727d75 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:14.877982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c7bdc-139b-455a-86b2-34c612727d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:14.877982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c7bdc-139b-455a-86b2-34c612727d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:14.879350 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6b6c7bdc-139b-455a-86b2-34c612727d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:14.978546 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c7bdc-139b-455a-86b2-34c612727d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:14.980041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c7bdc-139b-455a-86b2-34c612727d75 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:15.001749 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1373/2749] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:39:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:39:16.199446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f02dc5-966b-4d00-b0d5-c5a3399d98b8 service placement] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:16.200968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b6f02dc5-966b-4d00-b0d5-c5a3399d98b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:16.212754 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f02dc5-966b-4d00-b0d5-c5a3399d98b8 service placement] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:16.216928 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad29116-1e45-44fa-b96b-8bcd80be1e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:16.217524 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad29116-1e45-44fa-b96b-8bcd80be1e8f service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:16.217799 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad29116-1e45-44fa-b96b-8bcd80be1e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:16.218069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad29116-1e45-44fa-b96b-8bcd80be1e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:16.219891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6ad29116-1e45-44fa-b96b-8bcd80be1e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:16.234473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad29116-1e45-44fa-b96b-8bcd80be1e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:16.236047 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad29116-1e45-44fa-b96b-8bcd80be1e8f service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:16.247192 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1377/2750] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:39:24.998752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=46 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:26.242805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=6 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:35.985763 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c7bdc-139b-455a-86b2-34c612727d75 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:35.996258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c7bdc-139b-455a-86b2-34c612727d75 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:36.000912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6f83d-0b62-4849-9b3f-98260db4d0a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:36.001650 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6f83d-0b62-4849-9b3f-98260db4d0a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:36.002166 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6f83d-0b62-4849-9b3f-98260db4d0a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:36.002441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6f83d-0b62-4849-9b3f-98260db4d0a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:36.004228 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f1b6f83d-0b62-4849-9b3f-98260db4d0a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:36.019684 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6f83d-0b62-4849-9b3f-98260db4d0a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:36.021165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6f83d-0b62-4849-9b3f-98260db4d0a9 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:36.033574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1374/2751] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:39:40.921596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21cfc1-4631-48b6-b8be-654bc2d546be None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:40.922138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21cfc1-4631-48b6-b8be-654bc2d546be None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:40.922138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21cfc1-4631-48b6-b8be-654bc2d546be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:40.943791 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-bf21cfc1-4631-48b6-b8be-654bc2d546be None None] MFA Rules not processed for user `6c22d28f87c941afb1fad1cfaf3a5fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:39:40.945055 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-bf21cfc1-4631-48b6-b8be-654bc2d546be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:40.952274 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1378/2752] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:39:44.409863 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6f83d-0b62-4849-9b3f-98260db4d0a9 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:44.423683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f1b6f83d-0b62-4849-9b3f-98260db4d0a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:44.436275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6f83d-0b62-4849-9b3f-98260db4d0a9 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:44.441943 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:44.445289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:44.445289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:44.445289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] PATH_INFO: `/v3/users/e7a29212dfae4a93beea7d11085cde63` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:44.445289 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7a29212dfae4a93beea7d11085cde63)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:44.446658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:44.659612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7a29212dfae4a93beea7d11085cde63'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:44.660225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7a29212dfae4a93beea7d11085cde63'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:44.759958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7a29212dfae4a93beea7d11085cde63'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:44.774942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7a29212dfae4a93beea7d11085cde63'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:44.775722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7a29212dfae4a93beea7d11085cde63'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:44.789642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1375/2753] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:39:44 2022] DELETE /identity/v3/users/e7a29212dfae4a93beea7d11085cde63 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:39:45.070135 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21cfc1-4631-48b6-b8be-654bc2d546be None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:45.085102 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21cfc1-4631-48b6-b8be-654bc2d546be None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:45.160792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:45.161547 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:45.161822 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:45.162093 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] PATH_INFO: `/v3/projects/4377cb4a93524abd99acb6ebe07fd1d6` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:45.163273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4377cb4a93524abd99acb6ebe07fd1d6)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:45.165500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:45.302494 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4377cb4a93524abd99acb6ebe07fd1d6', 'request_id': 'req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:45.320622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4377cb4a93524abd99acb6ebe07fd1d6', 'request_id': 'req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:45.430158 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4377cb4a93524abd99acb6ebe07fd1d6', 'request_id': 'req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:45.443456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] The token cache is being invalidate because project 4377cb4a93524abd99acb6ebe07fd1d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:39:45.444542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:45.444792 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:45.446434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1379/2754] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:39:45 2022] DELETE /identity/v3/projects/4377cb4a93524abd99acb6ebe07fd1d6 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:39:45.451264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:45.453018 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:45.467220 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17fdfb-a561-4f3d-92ad-2dc757f10d60 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:45.535710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:45.536551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:45.536920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:45.537288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] PATH_INFO: `/v3/users/2e7a483ed0c84cc699dffdc4fc8fae08` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:45.538821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7a483ed0c84cc699dffdc4fc8fae08)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:45.541587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:45.764971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7a483ed0c84cc699dffdc4fc8fae08'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:45.765635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7a483ed0c84cc699dffdc4fc8fae08'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:45.856483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7a483ed0c84cc699dffdc4fc8fae08'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:45.863085 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:45.870181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7a483ed0c84cc699dffdc4fc8fae08'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:45.870877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7a483ed0c84cc699dffdc4fc8fae08'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:45.883497 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1376/2755] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:39:45 2022] DELETE /identity/v3/users/2e7a483ed0c84cc699dffdc4fc8fae08 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:39:46.109519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:46.111077 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:46.169707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb21a4-7cb1-4883-b236-a55365dc4ec5 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:46.179243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:39:46.181473 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:46.250447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee7dc0-d124-4fcc-97c6-68d3be440a98 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:39:46.255628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:46.256258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:46.256536 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:46.256831 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] PATH_INFO: `/v3/projects/ffc9db130a0043e98bc9e518812757b4` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:46.257955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc9db130a0043e98bc9e518812757b4)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:46.259890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:46.289006 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:46.300329 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b5fe4-5933-4b14-a6f5-ae2ada582898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:39:46.300977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b5fe4-5933-4b14-a6f5-ae2ada582898 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:39:46.301259 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b5fe4-5933-4b14-a6f5-ae2ada582898 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:39:46.301533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b5fe4-5933-4b14-a6f5-ae2ada582898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:39:46.303396 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-899b5fe4-5933-4b14-a6f5-ae2ada582898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:39:46.313168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b5fe4-5933-4b14-a6f5-ae2ada582898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:39:46.316114 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b5fe4-5933-4b14-a6f5-ae2ada582898 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:39:46.322763 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1380/2756] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:39:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:39:46.380660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc9db130a0043e98bc9e518812757b4', 'request_id': 'req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:46.386603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc9db130a0043e98bc9e518812757b4', 'request_id': 'req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:46.481364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc9db130a0043e98bc9e518812757b4', 'request_id': 'req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:46.493574 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] The token cache is being invalidate because project ffc9db130a0043e98bc9e518812757b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:39:46.493889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:46.494538 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-eb1542bf-c0e7-46cf-8a53-5df80d1118a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:39:46.496109 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1377/2757] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:39:46 2022] DELETE /identity/v3/projects/ffc9db130a0043e98bc9e518812757b4 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:39:56.612697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=60,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:39:56.939286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=8 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:40:21.534454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b5fe4-5933-4b14-a6f5-ae2ada582898 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:40:21.536063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-899b5fe4-5933-4b14-a6f5-ae2ada582898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:40:21.613961 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b5fe4-5933-4b14-a6f5-ae2ada582898 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:40:21.682544 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2c954-ba56-48f0-8a5a-1477505b6d6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:40:21.683227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2c954-ba56-48f0-8a5a-1477505b6d6b service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:40:21.683506 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2c954-ba56-48f0-8a5a-1477505b6d6b service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:40:21.683808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2c954-ba56-48f0-8a5a-1477505b6d6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:40:21.685621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-65c2c954-ba56-48f0-8a5a-1477505b6d6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:40:21.698925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2c954-ba56-48f0-8a5a-1477505b6d6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:40:21.700562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2c954-ba56-48f0-8a5a-1477505b6d6b service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:40:21.712117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1381/2758] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:40:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:40:31.707980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=62 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:40:37.877510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be741c4-b4e6-4a75-96f3-038d4711287a None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:40:37.877510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be741c4-b4e6-4a75-96f3-038d4711287a None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:40:37.877510 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be741c4-b4e6-4a75-96f3-038d4711287a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:40:37.896906 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-4be741c4-b4e6-4a75-96f3-038d4711287a None None] MFA Rules not processed for user `8bfb013575d540c59121ac8a1613fc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:40:37.898091 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4be741c4-b4e6-4a75-96f3-038d4711287a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:40:37.951652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1378/2759] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:40:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:40:39.940825 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2c954-ba56-48f0-8a5a-1477505b6d6b service nova] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:40:39.942228 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-65c2c954-ba56-48f0-8a5a-1477505b6d6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:40:40.025712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2c954-ba56-48f0-8a5a-1477505b6d6b service nova] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:40:40.030681 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850a501-23cf-4518-9a3c-187f64d8a5bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:40:40.031308 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850a501-23cf-4518-9a3c-187f64d8a5bb service placement] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:40:40.031976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850a501-23cf-4518-9a3c-187f64d8a5bb service placement] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:40:40.031976 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850a501-23cf-4518-9a3c-187f64d8a5bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:40:40.033989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0850a501-23cf-4518-9a3c-187f64d8a5bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:40:40.048903 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850a501-23cf-4518-9a3c-187f64d8a5bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:40:40.050845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850a501-23cf-4518-9a3c-187f64d8a5bb service placement] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:40:40.062239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1382/2760] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:40:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:40:47.945201 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=25 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:40:50.057527 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=32 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:40:50.681586 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be741c4-b4e6-4a75-96f3-038d4711287a None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:40:50.683058 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-4be741c4-b4e6-4a75-96f3-038d4711287a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:40:50.756417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be741c4-b4e6-4a75-96f3-038d4711287a None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:40:50.760889 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:40:50.761516 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:40:50.761792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:40:50.762062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] PATH_INFO: `/v3/users/6c22d28f87c941afb1fad1cfaf3a5fb2` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:40:50.763179 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c22d28f87c941afb1fad1cfaf3a5fb2)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:40:50.765147 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:40:50.959789 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c22d28f87c941afb1fad1cfaf3a5fb2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:40:50.960447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c22d28f87c941afb1fad1cfaf3a5fb2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:40:51.018101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c22d28f87c941afb1fad1cfaf3a5fb2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:40:51.032310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c22d28f87c941afb1fad1cfaf3a5fb2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:40:51.033014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c22d28f87c941afb1fad1cfaf3a5fb2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:40:51.045310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1379/2761] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:40:50 2022] DELETE /identity/v3/users/6c22d28f87c941afb1fad1cfaf3a5fb2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:40:51.317444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850a501-23cf-4518-9a3c-187f64d8a5bb service placement] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:40:51.329256 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850a501-23cf-4518-9a3c-187f64d8a5bb service placement] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:40:51.405997 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:40:51.407651 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:40:51.408144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:40:51.408668 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] PATH_INFO: `/v3/projects/2a77deb0bf54420e90c26e66699b3611` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:40:51.410619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a77deb0bf54420e90c26e66699b3611)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:40:51.414585 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:40:51.547794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a77deb0bf54420e90c26e66699b3611', 'request_id': 'req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:40:51.553234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a77deb0bf54420e90c26e66699b3611', 'request_id': 'req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:40:51.655963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a77deb0bf54420e90c26e66699b3611', 'request_id': 'req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:40:51.667414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] The token cache is being invalidate because project 2a77deb0bf54420e90c26e66699b3611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:40:51.667739 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:40:51.668365 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:40:51.669827 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1383/2762] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:40:51 2022] DELETE /identity/v3/projects/2a77deb0bf54420e90c26e66699b3611 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:40:52.510647 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:40:52.512348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:40:52.594656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82864e29-5aab-4f93-8e3b-95747500d646 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:40:52.608738 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:40:52.679205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7df66-ddd4-4c32-a06d-2f88f72dd96f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:40:52.679867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7df66-ddd4-4c32-a06d-2f88f72dd96f service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:40:52.680145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7df66-ddd4-4c32-a06d-2f88f72dd96f service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:40:52.680419 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7df66-ddd4-4c32-a06d-2f88f72dd96f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:40:52.682181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-28d7df66-ddd4-4c32-a06d-2f88f72dd96f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:40:52.695048 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7df66-ddd4-4c32-a06d-2f88f72dd96f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:40:52.696678 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7df66-ddd4-4c32-a06d-2f88f72dd96f service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:40:52.708092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1380/2763] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:40:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:41:01.664399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:41:03.051309 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=26 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:01.907803 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:01.909200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:01.965058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a1cbc-be4c-4e70-92f7-c2accc572b24 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:01.968621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfed46-5280-46b5-9679-d32146bcb4f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:01.969215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfed46-5280-46b5-9679-d32146bcb4f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:01.969490 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfed46-5280-46b5-9679-d32146bcb4f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:01.969759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfed46-5280-46b5-9679-d32146bcb4f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:01.971438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1fcfed46-5280-46b5-9679-d32146bcb4f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:02.028001 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfed46-5280-46b5-9679-d32146bcb4f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:02.029419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfed46-5280-46b5-9679-d32146bcb4f3 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:02.038269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1384/2764] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:12.034143 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=48 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:12.606184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7df66-ddd4-4c32-a06d-2f88f72dd96f service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:12.607623 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-28d7df66-ddd4-4c32-a06d-2f88f72dd96f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:12.664159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7df66-ddd4-4c32-a06d-2f88f72dd96f service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:12.668005 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0008bec-546b-4195-a960-4eeefa3f1014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:12.668601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0008bec-546b-4195-a960-4eeefa3f1014 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:12.668874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0008bec-546b-4195-a960-4eeefa3f1014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:12.669138 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0008bec-546b-4195-a960-4eeefa3f1014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:12.670816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d0008bec-546b-4195-a960-4eeefa3f1014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:12.733146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0008bec-546b-4195-a960-4eeefa3f1014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:12.734561 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0008bec-546b-4195-a960-4eeefa3f1014 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:12.744022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1381/2765] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:16.187760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfed46-5280-46b5-9679-d32146bcb4f3 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:16.189317 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1fcfed46-5280-46b5-9679-d32146bcb4f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:16.258971 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfed46-5280-46b5-9679-d32146bcb4f3 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:16.263967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cee8f2-a581-4834-8d01-5f426ea9297f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:16.265469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cee8f2-a581-4834-8d01-5f426ea9297f service placement] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:16.265469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cee8f2-a581-4834-8d01-5f426ea9297f service placement] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:16.265469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cee8f2-a581-4834-8d01-5f426ea9297f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:16.266963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-b8cee8f2-a581-4834-8d01-5f426ea9297f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:16.278932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cee8f2-a581-4834-8d01-5f426ea9297f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:16.280688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cee8f2-a581-4834-8d01-5f426ea9297f service placement] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:16.291018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1385/2766] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:16.320872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0008bec-546b-4195-a960-4eeefa3f1014 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:16.322491 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d0008bec-546b-4195-a960-4eeefa3f1014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:16.337688 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0008bec-546b-4195-a960-4eeefa3f1014 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:16.342325 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:16.346101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:16.346101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:16.346101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] PATH_INFO: `/v3/users/8bfb013575d540c59121ac8a1613fc0f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:16.346101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bfb013575d540c59121ac8a1613fc0f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:16.346532 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:16.529737 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bfb013575d540c59121ac8a1613fc0f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:16.530456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bfb013575d540c59121ac8a1613fc0f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:16.612417 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bfb013575d540c59121ac8a1613fc0f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:16.625233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bfb013575d540c59121ac8a1613fc0f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:16.625909 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-41c845e6-46bf-483a-b4b5-e0f268501bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bfb013575d540c59121ac8a1613fc0f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:16.639943 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1382/2767] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:42:16 2022] DELETE /identity/v3/users/8bfb013575d540c59121ac8a1613fc0f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:16.900666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cee8f2-a581-4834-8d01-5f426ea9297f service placement] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:16.912959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cee8f2-a581-4834-8d01-5f426ea9297f service placement] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:16.971845 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:16.972748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:16.973029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:16.973304 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] PATH_INFO: `/v3/projects/7b514c04a13742b1b19d46fec1c40f69` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:16.974457 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b514c04a13742b1b19d46fec1c40f69)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:16.976403 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:17.077591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b514c04a13742b1b19d46fec1c40f69', 'request_id': 'req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:17.083030 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b514c04a13742b1b19d46fec1c40f69', 'request_id': 'req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:17.166206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b514c04a13742b1b19d46fec1c40f69', 'request_id': 'req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:17.177706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] The token cache is being invalidate because project 7b514c04a13742b1b19d46fec1c40f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:42:17.180198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:17.180198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e3fb9962-6fd0-4d82-85e1-6724ce1739ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:17.180425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1386/2768] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:42:16 2022] DELETE /identity/v3/projects/7b514c04a13742b1b19d46fec1c40f69 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:17.185169 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29297e0-0117-44ba-9fa9-4f32ab5b6823 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.185453 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29297e0-0117-44ba-9fa9-4f32ab5b6823 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.185727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29297e0-0117-44ba-9fa9-4f32ab5b6823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:17.205886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d29297e0-0117-44ba-9fa9-4f32ab5b6823 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:17.207064 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d29297e0-0117-44ba-9fa9-4f32ab5b6823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:17.224233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:17.266933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1383/2769] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:17.274556 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b580b-6f81-4950-a538-0142f65ad333 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.275018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b580b-6f81-4950-a538-0142f65ad333 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.275117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b580b-6f81-4950-a538-0142f65ad333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:17.294915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-8c4b580b-6f81-4950-a538-0142f65ad333 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:17.296144 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8c4b580b-6f81-4950-a538-0142f65ad333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:17.304377 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1387/2770] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:17.308511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29297e0-0117-44ba-9fa9-4f32ab5b6823 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:17.317972 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29297e0-0117-44ba-9fa9-4f32ab5b6823 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:17.323698 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd019a18-0f03-4c3d-b720-c1a4beb36dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:17.324284 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd019a18-0f03-4c3d-b720-c1a4beb36dba admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.324599 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd019a18-0f03-4c3d-b720-c1a4beb36dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.324860 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd019a18-0f03-4c3d-b720-c1a4beb36dba admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:17.325500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd019a18-0f03-4c3d-b720-c1a4beb36dba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:42:17.325793 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd019a18-0f03-4c3d-b720-c1a4beb36dba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:17.327354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd019a18-0f03-4c3d-b720-c1a4beb36dba admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:17.338603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1384/2771] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:42:17.342907 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b580b-6f81-4950-a538-0142f65ad333 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:17.352334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b580b-6f81-4950-a538-0142f65ad333 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:17.357186 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237bd6c-1fd7-4bcb-91b7-b5a95f089b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:17.357780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237bd6c-1fd7-4bcb-91b7-b5a95f089b0d admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.358057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237bd6c-1fd7-4bcb-91b7-b5a95f089b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.358327 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237bd6c-1fd7-4bcb-91b7-b5a95f089b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:17.358977 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237bd6c-1fd7-4bcb-91b7-b5a95f089b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:17.360956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237bd6c-1fd7-4bcb-91b7-b5a95f089b0d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:17.372715 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:17.465071 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1388/2772] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:42:17 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:42:17.469200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd019a18-0f03-4c3d-b720-c1a4beb36dba admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:17.479145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd019a18-0f03-4c3d-b720-c1a4beb36dba admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:17.535883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056118a-afb9-47bc-a878-3acb23448f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:17.536527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056118a-afb9-47bc-a878-3acb23448f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.536822 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056118a-afb9-47bc-a878-3acb23448f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.537083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056118a-afb9-47bc-a878-3acb23448f39 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:17.537715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056118a-afb9-47bc-a878-3acb23448f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:17.539642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056118a-afb9-47bc-a878-3acb23448f39 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:17.606729 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1385/2773] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:42:17 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:42:17.610874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237bd6c-1fd7-4bcb-91b7-b5a95f089b0d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:17.621872 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237bd6c-1fd7-4bcb-91b7-b5a95f089b0d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:17.626814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5958c94-1530-4f3c-9f10-323a0b1691fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:17.627416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5958c94-1530-4f3c-9f10-323a0b1691fe admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.627738 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5958c94-1530-4f3c-9f10-323a0b1691fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.628017 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5958c94-1530-4f3c-9f10-323a0b1691fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:17.628725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5958c94-1530-4f3c-9f10-323a0b1691fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:17.630325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5958c94-1530-4f3c-9f10-323a0b1691fe admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:17.640140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1389/2774] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:42:17.644184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056118a-afb9-47bc-a878-3acb23448f39 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:17.654374 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056118a-afb9-47bc-a878-3acb23448f39 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:17.659275 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d623ffc-d0b2-414d-96ce-934b075aad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:17.660037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d623ffc-d0b2-414d-96ce-934b075aad08 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.660314 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d623ffc-d0b2-414d-96ce-934b075aad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.660582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d623ffc-d0b2-414d-96ce-934b075aad08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:17.661217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d623ffc-d0b2-414d-96ce-934b075aad08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:17.662836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d623ffc-d0b2-414d-96ce-934b075aad08 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:17.674149 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1386/2775] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:42:17.678064 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5958c94-1530-4f3c-9f10-323a0b1691fe admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:17.688613 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5958c94-1530-4f3c-9f10-323a0b1691fe admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:17.693229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5ec9a-63b4-4507-a939-ca0464534691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:17.693865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5ec9a-63b4-4507-a939-ca0464534691 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.694140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5ec9a-63b4-4507-a939-ca0464534691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.694434 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5ec9a-63b4-4507-a939-ca0464534691 admin admin] PATH_INFO: `/v3/projects/8bc09f98701c49d5a36a49e8001b5591/users/c772ccc9f9c14e41863827a4a5740077/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:17.708701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5ec9a-63b4-4507-a939-ca0464534691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc09f98701c49d5a36a49e8001b5591, user_id=c772ccc9f9c14e41863827a4a5740077, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:17.711318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5ec9a-63b4-4507-a939-ca0464534691 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:17.795980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1390/2776] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:42:17 2022] PUT /identity/v3/projects/8bc09f98701c49d5a36a49e8001b5591/users/c772ccc9f9c14e41863827a4a5740077/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:17.806890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b55fa-95cf-4e59-a3fd-84186b89ac61 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.807189 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b55fa-95cf-4e59-a3fd-84186b89ac61 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.807463 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b55fa-95cf-4e59-a3fd-84186b89ac61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:17.834913 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-528b55fa-95cf-4e59-a3fd-84186b89ac61 None None] MFA Rules not processed for user `c772ccc9f9c14e41863827a4a5740077`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:17.836135 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-528b55fa-95cf-4e59-a3fd-84186b89ac61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:17.902601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1387/2777] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:17.914318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5ec9a-63b4-4507-a939-ca0464534691 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:17.916140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-85a5ec9a-63b4-4507-a939-ca0464534691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:17.989852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5ec9a-63b4-4507-a939-ca0464534691 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:17.995700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06441e20-729f-4aa6-8396-09d2a966bc3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:17.996293 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06441e20-729f-4aa6-8396-09d2a966bc3c service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:17.996573 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06441e20-729f-4aa6-8396-09d2a966bc3c service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:17.996838 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06441e20-729f-4aa6-8396-09d2a966bc3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:18.004707 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06441e20-729f-4aa6-8396-09d2a966bc3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:18.006299 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06441e20-729f-4aa6-8396-09d2a966bc3c service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:18.014852 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1391/2778] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:18.234248 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b55fa-95cf-4e59-a3fd-84186b89ac61 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:18.253330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:18.297001 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b55fa-95cf-4e59-a3fd-84186b89ac61 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:18.301623 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:18.302180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:18.302500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:18.302858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] PATH_INFO: `/v3/users/c772ccc9f9c14e41863827a4a5740077` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:18.303992 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c772ccc9f9c14e41863827a4a5740077)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:18.305902 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:18.493740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c772ccc9f9c14e41863827a4a5740077'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:18.494432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c772ccc9f9c14e41863827a4a5740077'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:18.560962 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c772ccc9f9c14e41863827a4a5740077'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:18.574320 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c772ccc9f9c14e41863827a4a5740077'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:18.574977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c772ccc9f9c14e41863827a4a5740077'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:18.587644 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1388/2779] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:42:18 2022] DELETE /identity/v3/users/c772ccc9f9c14e41863827a4a5740077 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:18.592161 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29d63-8f26-4eec-b045-312a8c8e41cf None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:18.592643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29d63-8f26-4eec-b045-312a8c8e41cf None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:18.592746 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29d63-8f26-4eec-b045-312a8c8e41cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:18.611679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-c2b29d63-8f26-4eec-b045-312a8c8e41cf None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:18.612861 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c2b29d63-8f26-4eec-b045-312a8c8e41cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:18.670970 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1392/2780] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:18.680711 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:18.682150 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:18.738942 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532816b-ca73-43a5-a234-fe25338f91fe admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:18.743015 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8bb79-751f-47f6-921b-e9f4a9eecce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:18.743642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8bb79-751f-47f6-921b-e9f4a9eecce1 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:18.743914 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8bb79-751f-47f6-921b-e9f4a9eecce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:18.744180 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8bb79-751f-47f6-921b-e9f4a9eecce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:18.753413 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8bb79-751f-47f6-921b-e9f4a9eecce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:18.754809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8bb79-751f-47f6-921b-e9f4a9eecce1 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:18.764728 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1389/2781] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:19.164417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b29d63-8f26-4eec-b045-312a8c8e41cf None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:19.165906 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c2b29d63-8f26-4eec-b045-312a8c8e41cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:19.180704 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b29d63-8f26-4eec-b045-312a8c8e41cf None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:19.185490 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:19.186207 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:19.187021 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:19.187303 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] PATH_INFO: `/v3/projects/8bc09f98701c49d5a36a49e8001b5591` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:19.188697 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bc09f98701c49d5a36a49e8001b5591)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:19.190655 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:19.271750 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:19.286483 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bc09f98701c49d5a36a49e8001b5591', 'request_id': 'req-2307b426-d740-4776-9aeb-4d307b198fc9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:19.292423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bc09f98701c49d5a36a49e8001b5591', 'request_id': 'req-2307b426-d740-4776-9aeb-4d307b198fc9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:19.343332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bc09f98701c49d5a36a49e8001b5591', 'request_id': 'req-2307b426-d740-4776-9aeb-4d307b198fc9'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:19.354925 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] The token cache is being invalidate because project 8bc09f98701c49d5a36a49e8001b5591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:42:19.355209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:19.355912 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-2307b426-d740-4776-9aeb-4d307b198fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:19.357387 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1393/2782] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:42:19 2022] DELETE /identity/v3/projects/8bc09f98701c49d5a36a49e8001b5591 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:19.362409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ce983-5bf8-48bf-ba7e-90a47366ce20 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:19.362720 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ce983-5bf8-48bf-ba7e-90a47366ce20 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:19.362998 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ce983-5bf8-48bf-ba7e-90a47366ce20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:19.383515 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-830ce983-5bf8-48bf-ba7e-90a47366ce20 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:19.384778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-830ce983-5bf8-48bf-ba7e-90a47366ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:19.417715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:19.442462 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1390/2783] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:42:19.451321 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7e8ba-18fb-4717-ae26-0347d886b5ad None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:19.453595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7e8ba-18fb-4717-ae26-0347d886b5ad None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:19.453595 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7e8ba-18fb-4717-ae26-0347d886b5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:19.471760 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-46a7e8ba-18fb-4717-ae26-0347d886b5ad None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:19.472953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-46a7e8ba-18fb-4717-ae26-0347d886b5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:19.481712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1394/2784] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:19.486024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ce983-5bf8-48bf-ba7e-90a47366ce20 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:19.495805 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ce983-5bf8-48bf-ba7e-90a47366ce20 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:19.500430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a232ae-2752-4054-b49b-a13099899116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:19.501022 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a232ae-2752-4054-b49b-a13099899116 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:19.501297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a232ae-2752-4054-b49b-a13099899116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:19.501814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a232ae-2752-4054-b49b-a13099899116 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:19.502486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a232ae-2752-4054-b49b-a13099899116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:42:19.502786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a232ae-2752-4054-b49b-a13099899116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:19.504416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a232ae-2752-4054-b49b-a13099899116 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:19.515871 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1391/2785] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 12:42:19.519975 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7e8ba-18fb-4717-ae26-0347d886b5ad None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:19.529495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7e8ba-18fb-4717-ae26-0347d886b5ad None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:19.534255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf162e31-c3b1-4073-a742-8c07a23cc102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:19.534856 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf162e31-c3b1-4073-a742-8c07a23cc102 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:19.535138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf162e31-c3b1-4073-a742-8c07a23cc102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:19.535409 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf162e31-c3b1-4073-a742-8c07a23cc102 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:19.536076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf162e31-c3b1-4073-a742-8c07a23cc102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:19.538591 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf162e31-c3b1-4073-a742-8c07a23cc102 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:19.613525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1395/2786] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:42:19 2022] POST /identity/v3/projects => generated 403 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:42:19.617750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a232ae-2752-4054-b49b-a13099899116 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:19.627823 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a232ae-2752-4054-b49b-a13099899116 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:19.684202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529563e1-2b57-4eaf-b97d-232776d2b8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:19.684727 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529563e1-2b57-4eaf-b97d-232776d2b8dd admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:19.685004 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529563e1-2b57-4eaf-b97d-232776d2b8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:19.685276 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529563e1-2b57-4eaf-b97d-232776d2b8dd admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:19.685891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529563e1-2b57-4eaf-b97d-232776d2b8dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:19.687809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529563e1-2b57-4eaf-b97d-232776d2b8dd admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:19.771402 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1392/2787] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:42:19 2022] POST /identity/v3/users => generated 302 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:42:19.776147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf162e31-c3b1-4073-a742-8c07a23cc102 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:19.786685 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf162e31-c3b1-4073-a742-8c07a23cc102 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:19.792156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366f52b-a9e0-4615-b3fc-d99b615ba5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:19.792748 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366f52b-a9e0-4615-b3fc-d99b615ba5dd admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:19.793026 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366f52b-a9e0-4615-b3fc-d99b615ba5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:19.793301 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366f52b-a9e0-4615-b3fc-d99b615ba5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:19.794061 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366f52b-a9e0-4615-b3fc-d99b615ba5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:19.795756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366f52b-a9e0-4615-b3fc-d99b615ba5dd admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:19.805959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1396/2788] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:42:19.812408 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529563e1-2b57-4eaf-b97d-232776d2b8dd admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:19.822444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529563e1-2b57-4eaf-b97d-232776d2b8dd admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:19.827008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4e31b-c9aa-400d-9e22-672218766708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:19.827631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4e31b-c9aa-400d-9e22-672218766708 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:19.838598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4e31b-c9aa-400d-9e22-672218766708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:19.838598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4e31b-c9aa-400d-9e22-672218766708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:19.838598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4e31b-c9aa-400d-9e22-672218766708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:19.838598 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4e31b-c9aa-400d-9e22-672218766708 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:19.841219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1393/2789] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:42:19.845198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366f52b-a9e0-4615-b3fc-d99b615ba5dd admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:19.855937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366f52b-a9e0-4615-b3fc-d99b615ba5dd admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:19.860605 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104eb71-cd02-44d7-be56-dac49ca73ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:19.867641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104eb71-cd02-44d7-be56-dac49ca73ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:19.867641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104eb71-cd02-44d7-be56-dac49ca73ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:19.867641 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104eb71-cd02-44d7-be56-dac49ca73ef3 admin admin] PATH_INFO: `/v3/projects/92bc6043b43844bd985e66e940260ae3/users/b6ddfa04fef246d3aaabb797dbbe2284/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:19.876331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104eb71-cd02-44d7-be56-dac49ca73ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92bc6043b43844bd985e66e940260ae3, user_id=b6ddfa04fef246d3aaabb797dbbe2284, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:19.879271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104eb71-cd02-44d7-be56-dac49ca73ef3 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:19.950107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1397/2790] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:42:19 2022] PUT /identity/v3/projects/92bc6043b43844bd985e66e940260ae3/users/b6ddfa04fef246d3aaabb797dbbe2284/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:19.955014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b714d-63c8-4ac1-a9d5-8199493de1e7 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:19.955296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b714d-63c8-4ac1-a9d5-8199493de1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:19.955623 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b714d-63c8-4ac1-a9d5-8199493de1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:19.975093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-305b714d-63c8-4ac1-a9d5-8199493de1e7 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:19.976342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-305b714d-63c8-4ac1-a9d5-8199493de1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:20.034004 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1394/2791] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:20.043682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104eb71-cd02-44d7-be56-dac49ca73ef3 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:20.045367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-4104eb71-cd02-44d7-be56-dac49ca73ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:20.103503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104eb71-cd02-44d7-be56-dac49ca73ef3 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:20.107170 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470dbfc-e65c-4015-8ec9-bd389dd83359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:20.107809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470dbfc-e65c-4015-8ec9-bd389dd83359 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:20.108089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470dbfc-e65c-4015-8ec9-bd389dd83359 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:20.108361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470dbfc-e65c-4015-8ec9-bd389dd83359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:20.117187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470dbfc-e65c-4015-8ec9-bd389dd83359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:20.118638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470dbfc-e65c-4015-8ec9-bd389dd83359 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:20.128187 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1398/2792] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:21.304294 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8f428-f736-4c13-b091-caa747a5d2a0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:21.304851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8f428-f736-4c13-b091-caa747a5d2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:21.304851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8f428-f736-4c13-b091-caa747a5d2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:21.336012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-d7d8f428-f736-4c13-b091-caa747a5d2a0 None None] MFA Rules not processed for user `b6ddfa04fef246d3aaabb797dbbe2284`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:21.336012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d7d8f428-f736-4c13-b091-caa747a5d2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:21.371088 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:21.402796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1395/2793] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:21 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:21.407386 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470dbfc-e65c-4015-8ec9-bd389dd83359 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:21.417048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470dbfc-e65c-4015-8ec9-bd389dd83359 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:21.421594 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b24ea-7a1e-444b-8911-a9434f3e71ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:21.422200 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b24ea-7a1e-444b-8911-a9434f3e71ab admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:21.422474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b24ea-7a1e-444b-8911-a9434f3e71ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:21.422747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b24ea-7a1e-444b-8911-a9434f3e71ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:21.423372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b24ea-7a1e-444b-8911-a9434f3e71ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:21.425314 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b24ea-7a1e-444b-8911-a9434f3e71ab admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:21.533416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1399/2794] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:42:21 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:42:21.537637 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8f428-f736-4c13-b091-caa747a5d2a0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:21.548198 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8f428-f736-4c13-b091-caa747a5d2a0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:21.604991 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221164d-75d6-4565-9268-3169fcf6db42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:21.605601 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221164d-75d6-4565-9268-3169fcf6db42 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:21.605878 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221164d-75d6-4565-9268-3169fcf6db42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:21.606145 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221164d-75d6-4565-9268-3169fcf6db42 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:21.606778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221164d-75d6-4565-9268-3169fcf6db42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:21.608725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221164d-75d6-4565-9268-3169fcf6db42 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:21.680414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1396/2795] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:42:21 2022] POST /identity/v3/users => generated 309 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:42:21.684381 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b24ea-7a1e-444b-8911-a9434f3e71ab admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:21.695132 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b24ea-7a1e-444b-8911-a9434f3e71ab admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:21.699733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c685d3-ec21-47f5-ac71-922470ce9722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:21.700327 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c685d3-ec21-47f5-ac71-922470ce9722 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:21.700606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c685d3-ec21-47f5-ac71-922470ce9722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:21.700879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c685d3-ec21-47f5-ac71-922470ce9722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:21.701514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c685d3-ec21-47f5-ac71-922470ce9722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:21.703106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c685d3-ec21-47f5-ac71-922470ce9722 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:21.714189 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1400/2796] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:42:21.718268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221164d-75d6-4565-9268-3169fcf6db42 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:21.728288 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221164d-75d6-4565-9268-3169fcf6db42 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:21.732872 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97ca31-63cc-4f5a-87bf-5c1f2560cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:21.733512 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ca31-63cc-4f5a-87bf-5c1f2560cac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:21.733790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ca31-63cc-4f5a-87bf-5c1f2560cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:21.734062 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ca31-63cc-4f5a-87bf-5c1f2560cac3 admin admin] PATH_INFO: `/v3/projects/b9a7efbbcec44caf8b81af9db4bdc7ed/users/b0170cf76ea643a2b57ff12e2e45f59f/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:21.749070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97ca31-63cc-4f5a-87bf-5c1f2560cac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a7efbbcec44caf8b81af9db4bdc7ed, user_id=b0170cf76ea643a2b57ff12e2e45f59f, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:21.751987 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97ca31-63cc-4f5a-87bf-5c1f2560cac3 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:21.824439 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1397/2797] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:42:21 2022] PUT /identity/v3/projects/b9a7efbbcec44caf8b81af9db4bdc7ed/users/b0170cf76ea643a2b57ff12e2e45f59f/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:21.828477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c685d3-ec21-47f5-ac71-922470ce9722 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:21.838577 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c685d3-ec21-47f5-ac71-922470ce9722 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:21.856967 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:21.896334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd80750-82e3-4de7-8ec6-05321bc7ca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:21.896943 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd80750-82e3-4de7-8ec6-05321bc7ca0f admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:21.897226 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd80750-82e3-4de7-8ec6-05321bc7ca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:21.897502 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd80750-82e3-4de7-8ec6-05321bc7ca0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:21.898157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd80750-82e3-4de7-8ec6-05321bc7ca0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:21.899808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd80750-82e3-4de7-8ec6-05321bc7ca0f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:21.909286 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1401/2798] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:42:21.913441 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97ca31-63cc-4f5a-87bf-5c1f2560cac3 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:21.923766 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97ca31-63cc-4f5a-87bf-5c1f2560cac3 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:21.932225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27350332-9871-4115-9b83-e8d0f486f621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:21.932225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27350332-9871-4115-9b83-e8d0f486f621 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:21.932225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27350332-9871-4115-9b83-e8d0f486f621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:21.932225 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27350332-9871-4115-9b83-e8d0f486f621 admin admin] PATH_INFO: `/v3/projects/b9a7efbbcec44caf8b81af9db4bdc7ed/users/b0170cf76ea643a2b57ff12e2e45f59f/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:21.932638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27350332-9871-4115-9b83-e8d0f486f621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a7efbbcec44caf8b81af9db4bdc7ed, user_id=b0170cf76ea643a2b57ff12e2e45f59f, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:21.934544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27350332-9871-4115-9b83-e8d0f486f621 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:21.945252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1398/2799] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:42:21 2022] PUT /identity/v3/projects/b9a7efbbcec44caf8b81af9db4bdc7ed/users/b0170cf76ea643a2b57ff12e2e45f59f/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:23.134664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606a03b-be89-449d-af39-6cb8fc29c61b None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:23.137767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606a03b-be89-449d-af39-6cb8fc29c61b None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:23.137767 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606a03b-be89-449d-af39-6cb8fc29c61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:23.164535 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-e606a03b-be89-449d-af39-6cb8fc29c61b None None] MFA Rules not processed for user `b0170cf76ea643a2b57ff12e2e45f59f`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:23.165864 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e606a03b-be89-449d-af39-6cb8fc29c61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:23.238348 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1402/2800] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:23 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:24.527183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27350332-9871-4115-9b83-e8d0f486f621 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:24.591134 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27350332-9871-4115-9b83-e8d0f486f621 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:24.595627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:24.596243 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:24.596518 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:24.596787 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] PATH_INFO: `/v3/users/b6ddfa04fef246d3aaabb797dbbe2284` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:24.597898 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ddfa04fef246d3aaabb797dbbe2284)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:24.599827 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:24.715183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:24.727836 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ddfa04fef246d3aaabb797dbbe2284'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:24.728486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ddfa04fef246d3aaabb797dbbe2284'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:24.771712 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ddfa04fef246d3aaabb797dbbe2284'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:24.783455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ddfa04fef246d3aaabb797dbbe2284'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:24.784120 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ddfa04fef246d3aaabb797dbbe2284'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:24.797219 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1399/2801] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:42:24 2022] DELETE /identity/v3/users/b6ddfa04fef246d3aaabb797dbbe2284 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:25.002269 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606a03b-be89-449d-af39-6cb8fc29c61b None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:25.013366 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606a03b-be89-449d-af39-6cb8fc29c61b None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:25.073536 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:25.074171 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:25.074445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:25.074714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] PATH_INFO: `/v3/projects/92bc6043b43844bd985e66e940260ae3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:25.075849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92bc6043b43844bd985e66e940260ae3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:25.077784 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:25.169446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:25.172340 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92bc6043b43844bd985e66e940260ae3', 'request_id': 'req-00e333e0-1937-46d6-ab2e-2a12098b4894'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:25.177802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92bc6043b43844bd985e66e940260ae3', 'request_id': 'req-00e333e0-1937-46d6-ab2e-2a12098b4894'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:25.251865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92bc6043b43844bd985e66e940260ae3', 'request_id': 'req-00e333e0-1937-46d6-ab2e-2a12098b4894'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:25.267982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] The token cache is being invalidate because project 92bc6043b43844bd985e66e940260ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:42:25.268275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:25.268920 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:25.270501 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1403/2802] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:42:25 2022] DELETE /identity/v3/projects/92bc6043b43844bd985e66e940260ae3 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:42:25.274844 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:25.276357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:25.290039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de39b3-aca7-443c-bff0-fa54ed9080b7 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:25.346130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:25.346753 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:25.347028 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:25.347299 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] PATH_INFO: `/v3/users/b0170cf76ea643a2b57ff12e2e45f59f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:25.348451 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0170cf76ea643a2b57ff12e2e45f59f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:25.350348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:25.533900 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0170cf76ea643a2b57ff12e2e45f59f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:25.534652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0170cf76ea643a2b57ff12e2e45f59f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:25.600571 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0170cf76ea643a2b57ff12e2e45f59f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:25.613885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0170cf76ea643a2b57ff12e2e45f59f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:25.614527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e3f06c1e-33aa-4092-9808-45ed51d0f8ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0170cf76ea643a2b57ff12e2e45f59f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:25.627622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1400/2803] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:42:25 2022] DELETE /identity/v3/users/b0170cf76ea643a2b57ff12e2e45f59f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:25.859324 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:25.860809 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:25.873609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e333e0-1937-46d6-ab2e-2a12098b4894 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:25.935492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:25.936150 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:25.936427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:25.936700 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] PATH_INFO: `/v3/projects/b9a7efbbcec44caf8b81af9db4bdc7ed` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:25.937810 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a7efbbcec44caf8b81af9db4bdc7ed)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:25.939752 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:26.034423 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a7efbbcec44caf8b81af9db4bdc7ed', 'request_id': 'req-5ba64414-4e29-472c-be9b-b2f785e6fde0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:26.039884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a7efbbcec44caf8b81af9db4bdc7ed', 'request_id': 'req-5ba64414-4e29-472c-be9b-b2f785e6fde0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:26.097404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a7efbbcec44caf8b81af9db4bdc7ed', 'request_id': 'req-5ba64414-4e29-472c-be9b-b2f785e6fde0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:26.110180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] The token cache is being invalidate because project b9a7efbbcec44caf8b81af9db4bdc7ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:42:26.110470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:26.115517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5ba64414-4e29-472c-be9b-b2f785e6fde0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:42:26.115517 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1404/2804] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:42:25 2022] DELETE /identity/v3/projects/b9a7efbbcec44caf8b81af9db4bdc7ed => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:26.117544 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dd044-3d1a-4cb1-bd0f-35074aa810cd None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:26.117826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dd044-3d1a-4cb1-bd0f-35074aa810cd None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:26.118099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dd044-3d1a-4cb1-bd0f-35074aa810cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:26.138053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-7e3dd044-3d1a-4cb1-bd0f-35074aa810cd None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:26.139251 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7e3dd044-3d1a-4cb1-bd0f-35074aa810cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:26.160343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:26.199899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1401/2805] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:26.207599 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7aefc0-a75b-47ee-9c63-f7cc1b43056a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:26.207882 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7aefc0-a75b-47ee-9c63-f7cc1b43056a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:26.208153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7aefc0-a75b-47ee-9c63-f7cc1b43056a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:26.228411 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-0c7aefc0-a75b-47ee-9c63-f7cc1b43056a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:26.229618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-0c7aefc0-a75b-47ee-9c63-f7cc1b43056a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:26.237516 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1405/2806] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:42:26.242146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dd044-3d1a-4cb1-bd0f-35074aa810cd None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:26.252480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dd044-3d1a-4cb1-bd0f-35074aa810cd None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:26.257605 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c3b0a-520b-4d31-84f4-dc7c11261ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:26.258235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c3b0a-520b-4d31-84f4-dc7c11261ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:26.258509 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c3b0a-520b-4d31-84f4-dc7c11261ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:26.258777 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c3b0a-520b-4d31-84f4-dc7c11261ac9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:26.259422 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c3b0a-520b-4d31-84f4-dc7c11261ac9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:42:26.259757 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c3b0a-520b-4d31-84f4-dc7c11261ac9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:26.261331 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c3b0a-520b-4d31-84f4-dc7c11261ac9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:26.272636 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1402/2807] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:42:26.276629 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7aefc0-a75b-47ee-9c63-f7cc1b43056a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:26.285762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7aefc0-a75b-47ee-9c63-f7cc1b43056a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:26.290459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918e16e-097a-4462-8740-4707d8dc5c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:26.291045 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918e16e-097a-4462-8740-4707d8dc5c70 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:26.291325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918e16e-097a-4462-8740-4707d8dc5c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:26.291621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918e16e-097a-4462-8740-4707d8dc5c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:26.292249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918e16e-097a-4462-8740-4707d8dc5c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:26.294124 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918e16e-097a-4462-8740-4707d8dc5c70 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:26.391037 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1406/2808] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:42:26 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:42:26.395101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c3b0a-520b-4d31-84f4-dc7c11261ac9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:26.405245 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c3b0a-520b-4d31-84f4-dc7c11261ac9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:26.461208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ab245-1808-43c7-b70e-e33fccb22f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:26.461812 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ab245-1808-43c7-b70e-e33fccb22f0c admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:26.462093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ab245-1808-43c7-b70e-e33fccb22f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:26.462369 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ab245-1808-43c7-b70e-e33fccb22f0c admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:26.463002 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ab245-1808-43c7-b70e-e33fccb22f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:26.464971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ab245-1808-43c7-b70e-e33fccb22f0c admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:26.548465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1403/2809] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:42:26 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:42:26.552745 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918e16e-097a-4462-8740-4707d8dc5c70 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:26.563519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918e16e-097a-4462-8740-4707d8dc5c70 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:26.568204 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399c2e9-7e09-48a9-be96-849915aebf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:26.568796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c2e9-7e09-48a9-be96-849915aebf9e admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:26.569078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c2e9-7e09-48a9-be96-849915aebf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:26.569980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c2e9-7e09-48a9-be96-849915aebf9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:26.570620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399c2e9-7e09-48a9-be96-849915aebf9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:26.572263 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399c2e9-7e09-48a9-be96-849915aebf9e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:26.582029 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1407/2810] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:42:26.585910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ab245-1808-43c7-b70e-e33fccb22f0c admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:26.595875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ab245-1808-43c7-b70e-e33fccb22f0c admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:26.601008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbea654-72be-4101-bcd6-066ad4a2692f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:26.601592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea654-72be-4101-bcd6-066ad4a2692f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:26.601865 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea654-72be-4101-bcd6-066ad4a2692f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:26.602131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea654-72be-4101-bcd6-066ad4a2692f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:26.602788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbea654-72be-4101-bcd6-066ad4a2692f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:26.604396 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbea654-72be-4101-bcd6-066ad4a2692f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:26.614723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1404/2811] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:42:26.619472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399c2e9-7e09-48a9-be96-849915aebf9e admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:26.629728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399c2e9-7e09-48a9-be96-849915aebf9e admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:26.634518 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c08de-d775-42eb-ac40-ba0fbda70863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:26.635162 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c08de-d775-42eb-ac40-ba0fbda70863 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:26.635451 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c08de-d775-42eb-ac40-ba0fbda70863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:26.635759 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c08de-d775-42eb-ac40-ba0fbda70863 admin admin] PATH_INFO: `/v3/projects/1482b534717943e48b1f103a3961cf18/users/72c580c20bf7440db7cc145d7230e42f/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:26.650051 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c08de-d775-42eb-ac40-ba0fbda70863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1482b534717943e48b1f103a3961cf18, user_id=72c580c20bf7440db7cc145d7230e42f, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:26.652749 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c08de-d775-42eb-ac40-ba0fbda70863 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:26.715813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:26.720375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1408/2812] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:42:26 2022] PUT /identity/v3/projects/1482b534717943e48b1f103a3961cf18/users/72c580c20bf7440db7cc145d7230e42f/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:26.737265 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557fe52-2acf-48df-a716-7071b69f9013 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:26.737552 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557fe52-2acf-48df-a716-7071b69f9013 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:26.737860 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557fe52-2acf-48df-a716-7071b69f9013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:26.766146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-5557fe52-2acf-48df-a716-7071b69f9013 None None] MFA Rules not processed for user `72c580c20bf7440db7cc145d7230e42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:26.767344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-5557fe52-2acf-48df-a716-7071b69f9013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:26.832277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1405/2813] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:42:26.837014 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c08de-d775-42eb-ac40-ba0fbda70863 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:26.898514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c08de-d775-42eb-ac40-ba0fbda70863 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:26.903239 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded2a0e-dc32-40e1-a2fd-8caba191dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:26.903795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded2a0e-dc32-40e1-a2fd-8caba191dea8 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:26.904072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded2a0e-dc32-40e1-a2fd-8caba191dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:26.904341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded2a0e-dc32-40e1-a2fd-8caba191dea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:26.904958 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded2a0e-dc32-40e1-a2fd-8caba191dea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:26.906843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded2a0e-dc32-40e1-a2fd-8caba191dea8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:27.001969 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1409/2814] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:42:26 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:42:27.005818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557fe52-2acf-48df-a716-7071b69f9013 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:27.016551 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557fe52-2acf-48df-a716-7071b69f9013 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:27.034618 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:27.073434 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287b6bb-97f6-493d-b47f-a238aa1aa1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:27.074069 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b6bb-97f6-493d-b47f-a238aa1aa1bb admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:27.074384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b6bb-97f6-493d-b47f-a238aa1aa1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:27.074655 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b6bb-97f6-493d-b47f-a238aa1aa1bb admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:27.075324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287b6bb-97f6-493d-b47f-a238aa1aa1bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:27.077279 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287b6bb-97f6-493d-b47f-a238aa1aa1bb admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:27.183835 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1406/2815] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:42:27 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:42:27.187621 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded2a0e-dc32-40e1-a2fd-8caba191dea8 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:27.198885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded2a0e-dc32-40e1-a2fd-8caba191dea8 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:27.203733 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286d647-0c09-4554-a007-1ac83e4e05aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:27.210352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286d647-0c09-4554-a007-1ac83e4e05aa admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:27.210352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286d647-0c09-4554-a007-1ac83e4e05aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:27.210352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286d647-0c09-4554-a007-1ac83e4e05aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:27.210352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286d647-0c09-4554-a007-1ac83e4e05aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:27.210352 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286d647-0c09-4554-a007-1ac83e4e05aa admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:27.222118 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1410/2816] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:42:27.223572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287b6bb-97f6-493d-b47f-a238aa1aa1bb admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:27.234296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287b6bb-97f6-493d-b47f-a238aa1aa1bb admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:27.238979 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864db1c-bd20-40de-adf6-2503723a9e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:27.239656 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864db1c-bd20-40de-adf6-2503723a9e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:27.239932 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864db1c-bd20-40de-adf6-2503723a9e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:27.240202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864db1c-bd20-40de-adf6-2503723a9e0f admin admin] PATH_INFO: `/v3/projects/1328597385724eae9725b1cab96a83af/users/54b38099c93c49c0bdbaa2a8ac4d857c/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:27.255258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864db1c-bd20-40de-adf6-2503723a9e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1328597385724eae9725b1cab96a83af, user_id=54b38099c93c49c0bdbaa2a8ac4d857c, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:27.257953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864db1c-bd20-40de-adf6-2503723a9e0f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:27.372316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1407/2817] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:42:27 2022] PUT /identity/v3/projects/1328597385724eae9725b1cab96a83af/users/54b38099c93c49c0bdbaa2a8ac4d857c/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:27.376214 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286d647-0c09-4554-a007-1ac83e4e05aa admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:27.387193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286d647-0c09-4554-a007-1ac83e4e05aa admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:27.442713 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf13dd-a9a6-44d0-8820-51539a80ad3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:27.443245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf13dd-a9a6-44d0-8820-51539a80ad3a admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:27.443526 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf13dd-a9a6-44d0-8820-51539a80ad3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:27.443834 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf13dd-a9a6-44d0-8820-51539a80ad3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:27.444955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf13dd-a9a6-44d0-8820-51539a80ad3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:27.446551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf13dd-a9a6-44d0-8820-51539a80ad3a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:27.456130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1411/2818] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:42:27.460828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864db1c-bd20-40de-adf6-2503723a9e0f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:27.471093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864db1c-bd20-40de-adf6-2503723a9e0f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:27.475788 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59a9cb-867a-4afc-ae2f-8f0cde3cf0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:27.479527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59a9cb-867a-4afc-ae2f-8f0cde3cf0a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:27.479527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59a9cb-867a-4afc-ae2f-8f0cde3cf0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:27.479527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59a9cb-867a-4afc-ae2f-8f0cde3cf0a1 admin admin] PATH_INFO: `/v3/projects/1328597385724eae9725b1cab96a83af/users/54b38099c93c49c0bdbaa2a8ac4d857c/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:27.479527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59a9cb-867a-4afc-ae2f-8f0cde3cf0a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1328597385724eae9725b1cab96a83af, user_id=54b38099c93c49c0bdbaa2a8ac4d857c, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:27.481527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59a9cb-867a-4afc-ae2f-8f0cde3cf0a1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:27.492177 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1408/2819] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:42:27 2022] PUT /identity/v3/projects/1328597385724eae9725b1cab96a83af/users/54b38099c93c49c0bdbaa2a8ac4d857c/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:42:27.503600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27676495-a757-4ce9-92a7-c66e7196fa25 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:27.503884 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27676495-a757-4ce9-92a7-c66e7196fa25 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:27.504157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27676495-a757-4ce9-92a7-c66e7196fa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:27.532772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-27676495-a757-4ce9-92a7-c66e7196fa25 None None] MFA Rules not processed for user `54b38099c93c49c0bdbaa2a8ac4d857c`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:42:27.533980 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-27676495-a757-4ce9-92a7-c66e7196fa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:27.595519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:27.605153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1412/2820] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:42:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:42:27.670409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59a9cb-867a-4afc-ae2f-8f0cde3cf0a1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:27.672235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-dc59a9cb-867a-4afc-ae2f-8f0cde3cf0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:27.732317 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59a9cb-867a-4afc-ae2f-8f0cde3cf0a1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:27.735841 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afc92d-c381-4819-8086-41d2cc263013 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:27.736560 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afc92d-c381-4819-8086-41d2cc263013 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:27.736828 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afc92d-c381-4819-8086-41d2cc263013 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:27.737094 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afc92d-c381-4819-8086-41d2cc263013 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:27.793059 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05afc92d-c381-4819-8086-41d2cc263013 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:27.794466 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05afc92d-c381-4819-8086-41d2cc263013 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:27.804008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1409/2821] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:33.969730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27676495-a757-4ce9-92a7-c66e7196fa25 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:42:33.971145 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-27676495-a757-4ce9-92a7-c66e7196fa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:42:33.983254 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27676495-a757-4ce9-92a7-c66e7196fa25 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:42:33.986770 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cad84a-5cf7-40c4-9639-905c658f0d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:42:33.987361 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cad84a-5cf7-40c4-9639-905c658f0d0f service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:42:33.987674 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cad84a-5cf7-40c4-9639-905c658f0d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:42:33.987944 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cad84a-5cf7-40c4-9639-905c658f0d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:42:33.997345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cad84a-5cf7-40c4-9639-905c658f0d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:42:33.998756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cad84a-5cf7-40c4-9639-905c658f0d0f service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:42:34.009104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1413/2822] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:42:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:42:37.799389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:42:44.004218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=8 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:01.144982 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afc92d-c381-4819-8086-41d2cc263013 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:01.207875 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afc92d-c381-4819-8086-41d2cc263013 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:01.212390 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:01.220966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:01.220966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:01.220966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] PATH_INFO: `/v3/users/72c580c20bf7440db7cc145d7230e42f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:01.220966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72c580c20bf7440db7cc145d7230e42f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:01.220966 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:01.361060 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72c580c20bf7440db7cc145d7230e42f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:01.361642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72c580c20bf7440db7cc145d7230e42f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:01.421455 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72c580c20bf7440db7cc145d7230e42f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:01.433679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72c580c20bf7440db7cc145d7230e42f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:01.436547 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72c580c20bf7440db7cc145d7230e42f'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:01.446283 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1410/2823] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:43:01 2022] DELETE /identity/v3/users/72c580c20bf7440db7cc145d7230e42f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:01.450808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95062a21-eeb5-4f37-aff6-fda9fb5e4c77 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:01.451273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95062a21-eeb5-4f37-aff6-fda9fb5e4c77 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:01.451400 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95062a21-eeb5-4f37-aff6-fda9fb5e4c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:01.471661 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-95062a21-eeb5-4f37-aff6-fda9fb5e4c77 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:01.472867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-95062a21-eeb5-4f37-aff6-fda9fb5e4c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:01.531174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1414/2824] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:01.541606 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:01.542986 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:01.601612 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb62298-4cb9-4dec-8611-776ad636f073 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:01.605880 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539f6f1-c5c3-4ab2-9ae9-cc865023dd37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:01.610732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539f6f1-c5c3-4ab2-9ae9-cc865023dd37 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:01.610732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539f6f1-c5c3-4ab2-9ae9-cc865023dd37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:01.610732 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539f6f1-c5c3-4ab2-9ae9-cc865023dd37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:01.616252 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539f6f1-c5c3-4ab2-9ae9-cc865023dd37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:01.617680 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539f6f1-c5c3-4ab2-9ae9-cc865023dd37 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:01.627621 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1411/2825] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:02.035062 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95062a21-eeb5-4f37-aff6-fda9fb5e4c77 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:02.036609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-95062a21-eeb5-4f37-aff6-fda9fb5e4c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:02.052211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95062a21-eeb5-4f37-aff6-fda9fb5e4c77 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:02.056851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:02.057493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:02.057775 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:02.058052 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] PATH_INFO: `/v3/projects/1482b534717943e48b1f103a3961cf18` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:02.059223 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1482b534717943e48b1f103a3961cf18)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:02.061198 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:02.170044 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1482b534717943e48b1f103a3961cf18', 'request_id': 'req-8a812aa0-95e7-409b-89ce-97453a691ee8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:02.175326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1482b534717943e48b1f103a3961cf18', 'request_id': 'req-8a812aa0-95e7-409b-89ce-97453a691ee8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:02.250174 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1482b534717943e48b1f103a3961cf18', 'request_id': 'req-8a812aa0-95e7-409b-89ce-97453a691ee8'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:02.262362 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] The token cache is being invalidate because project 1482b534717943e48b1f103a3961cf18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:43:02.262666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:02.263289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:02.264802 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1415/2826] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:43:02 2022] DELETE /identity/v3/projects/1482b534717943e48b1f103a3961cf18 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:02.268931 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539f6f1-c5c3-4ab2-9ae9-cc865023dd37 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:02.278576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539f6f1-c5c3-4ab2-9ae9-cc865023dd37 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:02.296316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:02.334985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:02.335638 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:02.335912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:02.336181 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] PATH_INFO: `/v3/users/54b38099c93c49c0bdbaa2a8ac4d857c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:02.337316 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] RBAC: Authorizing `identity:delete_user(user_id=54b38099c93c49c0bdbaa2a8ac4d857c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:02.339222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:02.514949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54b38099c93c49c0bdbaa2a8ac4d857c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:02.515511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54b38099c93c49c0bdbaa2a8ac4d857c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:02.587019 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54b38099c93c49c0bdbaa2a8ac4d857c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:02.599264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54b38099c93c49c0bdbaa2a8ac4d857c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:02.599936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-91824a26-c2fd-46e2-a289-3da645f3208e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54b38099c93c49c0bdbaa2a8ac4d857c'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:02.612529 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1412/2827] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:43:02 2022] DELETE /identity/v3/users/54b38099c93c49c0bdbaa2a8ac4d857c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:03.006454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:03.007998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:03.022915 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a812aa0-95e7-409b-89ce-97453a691ee8 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:03.080648 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:03.081285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:03.081564 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:03.081837 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] PATH_INFO: `/v3/projects/1328597385724eae9725b1cab96a83af` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:03.083072 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1328597385724eae9725b1cab96a83af)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:03.084909 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:03.183887 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:03.200367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1328597385724eae9725b1cab96a83af', 'request_id': 'req-00b822a1-e4f0-456c-a11d-614046685f00'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:03.205844 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1328597385724eae9725b1cab96a83af', 'request_id': 'req-00b822a1-e4f0-456c-a11d-614046685f00'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:03.273169 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1328597385724eae9725b1cab96a83af', 'request_id': 'req-00b822a1-e4f0-456c-a11d-614046685f00'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:03.284224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] The token cache is being invalidate because project 1328597385724eae9725b1cab96a83af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:43:03.284620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:03.285798 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-00b822a1-e4f0-456c-a11d-614046685f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:03.287332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1416/2828] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:43:03 2022] DELETE /identity/v3/projects/1328597385724eae9725b1cab96a83af => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:03.292611 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5006f0-6124-429a-98ff-238bb58f8570 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:03.304498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5006f0-6124-429a-98ff-238bb58f8570 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:03.304498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5006f0-6124-429a-98ff-238bb58f8570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:03.313159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-6d5006f0-6124-429a-98ff-238bb58f8570 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:03.314348 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-6d5006f0-6124-429a-98ff-238bb58f8570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:03.373485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1413/2829] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:43:03.381117 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd240b85-af6b-4b0c-a1bf-44ba5d42a305 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:03.381397 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd240b85-af6b-4b0c-a1bf-44ba5d42a305 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:03.381672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd240b85-af6b-4b0c-a1bf-44ba5d42a305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:03.400863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-bd240b85-af6b-4b0c-a1bf-44ba5d42a305 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:03.402060 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-bd240b85-af6b-4b0c-a1bf-44ba5d42a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:03.409772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1417/2830] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:43:03.413773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5006f0-6124-429a-98ff-238bb58f8570 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:03.423832 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5006f0-6124-429a-98ff-238bb58f8570 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:03.428349 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8122767-a9ee-452a-a8b6-7026610df1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:03.428940 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122767-a9ee-452a-a8b6-7026610df1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:03.429216 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122767-a9ee-452a-a8b6-7026610df1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:03.429485 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122767-a9ee-452a-a8b6-7026610df1d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:03.430126 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8122767-a9ee-452a-a8b6-7026610df1d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:43:03.430426 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8122767-a9ee-452a-a8b6-7026610df1d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:03.432619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8122767-a9ee-452a-a8b6-7026610df1d1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:03.443981 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1414/2831] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:43:03.448194 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd240b85-af6b-4b0c-a1bf-44ba5d42a305 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:03.457414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd240b85-af6b-4b0c-a1bf-44ba5d42a305 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:03.462130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711aaa6-c78b-41bc-b32e-eeb9a27eb0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:03.462728 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711aaa6-c78b-41bc-b32e-eeb9a27eb0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:03.463008 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711aaa6-c78b-41bc-b32e-eeb9a27eb0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:03.463280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711aaa6-c78b-41bc-b32e-eeb9a27eb0e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:03.463953 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711aaa6-c78b-41bc-b32e-eeb9a27eb0e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:03.465848 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711aaa6-c78b-41bc-b32e-eeb9a27eb0e9 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:03.571782 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1418/2832] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:43:03 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:03.575861 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8122767-a9ee-452a-a8b6-7026610df1d1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:03.586087 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8122767-a9ee-452a-a8b6-7026610df1d1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:03.614311 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:03.643708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e7458-3854-45c0-9ccc-9800105bb39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:03.644305 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7458-3854-45c0-9ccc-9800105bb39f admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:03.644580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7458-3854-45c0-9ccc-9800105bb39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:03.644849 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7458-3854-45c0-9ccc-9800105bb39f admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:03.645483 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e7458-3854-45c0-9ccc-9800105bb39f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:03.647368 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e7458-3854-45c0-9ccc-9800105bb39f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:03.770731 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1415/2833] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:43:03 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:03.774607 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711aaa6-c78b-41bc-b32e-eeb9a27eb0e9 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:03.785325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711aaa6-c78b-41bc-b32e-eeb9a27eb0e9 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:03.789843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10019529-c000-4d09-8c9c-59680cd88400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:03.790433 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10019529-c000-4d09-8c9c-59680cd88400 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:03.791448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10019529-c000-4d09-8c9c-59680cd88400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:03.791703 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10019529-c000-4d09-8c9c-59680cd88400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:03.797718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10019529-c000-4d09-8c9c-59680cd88400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:03.797718 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10019529-c000-4d09-8c9c-59680cd88400 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:03.803776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1419/2834] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:03.807704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e7458-3854-45c0-9ccc-9800105bb39f admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:03.820409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e7458-3854-45c0-9ccc-9800105bb39f admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:03.825131 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9726e-d99f-442a-82f0-7de13a48da6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:03.825867 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9726e-d99f-442a-82f0-7de13a48da6d admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:03.826214 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9726e-d99f-442a-82f0-7de13a48da6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:03.826559 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9726e-d99f-442a-82f0-7de13a48da6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:03.827226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9726e-d99f-442a-82f0-7de13a48da6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:03.828851 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9726e-d99f-442a-82f0-7de13a48da6d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:03.840130 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1416/2835] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:43:03.844438 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10019529-c000-4d09-8c9c-59680cd88400 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:03.855311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10019529-c000-4d09-8c9c-59680cd88400 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:03.859921 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f31278-9dfa-45e8-a9e2-3a74b15a8d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:03.860575 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f31278-9dfa-45e8-a9e2-3a74b15a8d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:03.860897 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f31278-9dfa-45e8-a9e2-3a74b15a8d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:03.861178 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f31278-9dfa-45e8-a9e2-3a74b15a8d3a admin admin] PATH_INFO: `/v3/projects/9d86903c504d4e388ee66df975eef99f/users/f4e9ae1c32f44c12a040950121a38ab3/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:03.875478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f31278-9dfa-45e8-a9e2-3a74b15a8d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d86903c504d4e388ee66df975eef99f, user_id=f4e9ae1c32f44c12a040950121a38ab3, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:03.878461 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f31278-9dfa-45e8-a9e2-3a74b15a8d3a admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:03.958553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1420/2836] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:03 2022] PUT /identity/v3/projects/9d86903c504d4e388ee66df975eef99f/users/f4e9ae1c32f44c12a040950121a38ab3/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:03.970391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd4268-b9d5-413c-876f-1d87ae61f649 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:03.970672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd4268-b9d5-413c-876f-1d87ae61f649 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:03.973102 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd4268-b9d5-413c-876f-1d87ae61f649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:03.998846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-90fd4268-b9d5-413c-876f-1d87ae61f649 None None] MFA Rules not processed for user `f4e9ae1c32f44c12a040950121a38ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:04.000082 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-90fd4268-b9d5-413c-876f-1d87ae61f649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:04.063885 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1417/2837] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:04.068661 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f31278-9dfa-45e8-a9e2-3a74b15a8d3a admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:04.132209 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f31278-9dfa-45e8-a9e2-3a74b15a8d3a admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:04.137473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe296f9c-bad8-405d-b904-69c3cc82e899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:04.138078 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296f9c-bad8-405d-b904-69c3cc82e899 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:04.138357 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296f9c-bad8-405d-b904-69c3cc82e899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:04.138630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296f9c-bad8-405d-b904-69c3cc82e899 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:04.139255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe296f9c-bad8-405d-b904-69c3cc82e899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:04.141261 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe296f9c-bad8-405d-b904-69c3cc82e899 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:04.255574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1421/2838] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:43:04 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:43:04.256968 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd4268-b9d5-413c-876f-1d87ae61f649 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:04.268269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd4268-b9d5-413c-876f-1d87ae61f649 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:04.323432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a1c3f-742c-4c02-bc99-d1b628671330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:04.324068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a1c3f-742c-4c02-bc99-d1b628671330 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:04.324313 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a1c3f-742c-4c02-bc99-d1b628671330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:04.324580 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a1c3f-742c-4c02-bc99-d1b628671330 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:04.325200 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a1c3f-742c-4c02-bc99-d1b628671330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:04.327107 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a1c3f-742c-4c02-bc99-d1b628671330 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:04.502826 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1418/2839] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:43:04 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:04.506805 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe296f9c-bad8-405d-b904-69c3cc82e899 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:04.518166 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe296f9c-bad8-405d-b904-69c3cc82e899 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:04.523389 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f87f4-13b6-4db4-8519-38cd0865934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:04.524058 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f87f4-13b6-4db4-8519-38cd0865934e admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:04.524333 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f87f4-13b6-4db4-8519-38cd0865934e admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:04.524622 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f87f4-13b6-4db4-8519-38cd0865934e admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:04.525234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f87f4-13b6-4db4-8519-38cd0865934e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:04.527460 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f87f4-13b6-4db4-8519-38cd0865934e admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:04.537492 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1422/2840] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:04.541739 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a1c3f-742c-4c02-bc99-d1b628671330 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:04.551595 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a1c3f-742c-4c02-bc99-d1b628671330 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:04.556080 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf21dc6-10c1-452b-a0f1-99530cefe006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:04.557208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf21dc6-10c1-452b-a0f1-99530cefe006 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:04.557481 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf21dc6-10c1-452b-a0f1-99530cefe006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:04.557745 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf21dc6-10c1-452b-a0f1-99530cefe006 admin admin] PATH_INFO: `/v3/projects/82f9c0de5ce9435a92e4e59d88df138c/users/e206ea197b364fcf86988d9ecd7e1396/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:04.572810 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf21dc6-10c1-452b-a0f1-99530cefe006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f9c0de5ce9435a92e4e59d88df138c, user_id=e206ea197b364fcf86988d9ecd7e1396, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:04.575412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf21dc6-10c1-452b-a0f1-99530cefe006 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:04.690415 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1419/2841] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:04 2022] PUT /identity/v3/projects/82f9c0de5ce9435a92e4e59d88df138c/users/e206ea197b364fcf86988d9ecd7e1396/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:04.694405 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f87f4-13b6-4db4-8519-38cd0865934e admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:04.704973 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f87f4-13b6-4db4-8519-38cd0865934e admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:04.724119 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:04.761498 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12eded-48d1-42cb-9d69-68076db5282d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:04.762032 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12eded-48d1-42cb-9d69-68076db5282d admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:04.762310 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12eded-48d1-42cb-9d69-68076db5282d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:04.762581 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12eded-48d1-42cb-9d69-68076db5282d admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:04.763208 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12eded-48d1-42cb-9d69-68076db5282d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:04.765130 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12eded-48d1-42cb-9d69-68076db5282d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:04.774820 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1423/2842] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:04.779092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf21dc6-10c1-452b-a0f1-99530cefe006 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:04.784723 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:04.789984 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf21dc6-10c1-452b-a0f1-99530cefe006 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:04.795158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7df4a-4077-4cb4-b750-e9e0f5b1fb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:04.795847 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7df4a-4077-4cb4-b750-e9e0f5b1fb90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:04.796122 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7df4a-4077-4cb4-b750-e9e0f5b1fb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:04.796391 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7df4a-4077-4cb4-b750-e9e0f5b1fb90 admin admin] PATH_INFO: `/v3/projects/82f9c0de5ce9435a92e4e59d88df138c/users/e206ea197b364fcf86988d9ecd7e1396/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:04.798395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7df4a-4077-4cb4-b750-e9e0f5b1fb90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f9c0de5ce9435a92e4e59d88df138c, user_id=e206ea197b364fcf86988d9ecd7e1396, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:04.801031 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7df4a-4077-4cb4-b750-e9e0f5b1fb90 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:04.811564 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1420/2843] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:04 2022] PUT /identity/v3/projects/82f9c0de5ce9435a92e4e59d88df138c/users/e206ea197b364fcf86988d9ecd7e1396/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:04.821609 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02b907-fce8-4e01-a96f-e00197b866c3 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:04.823156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02b907-fce8-4e01-a96f-e00197b866c3 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:04.823156 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02b907-fce8-4e01-a96f-e00197b866c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:04.850222 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-3b02b907-fce8-4e01-a96f-e00197b866c3 None None] MFA Rules not processed for user `e206ea197b364fcf86988d9ecd7e1396`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:04.851427 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3b02b907-fce8-4e01-a96f-e00197b866c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:04.924918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1424/2844] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:43:04.937044 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7df4a-4077-4cb4-b750-e9e0f5b1fb90 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:04.938740 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-59f7df4a-4077-4cb4-b750-e9e0f5b1fb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:04.995479 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7df4a-4077-4cb4-b750-e9e0f5b1fb90 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:04.999099 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b601c-1f65-4801-afb4-26d4c8dbd3fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:04.999730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b601c-1f65-4801-afb4-26d4c8dbd3fc service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:05.000008 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b601c-1f65-4801-afb4-26d4c8dbd3fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:05.000277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b601c-1f65-4801-afb4-26d4c8dbd3fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:05.009384 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b601c-1f65-4801-afb4-26d4c8dbd3fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:05.010790 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b601c-1f65-4801-afb4-26d4c8dbd3fc service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:05.020164 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1421/2845] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:05.088242 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02b907-fce8-4e01-a96f-e00197b866c3 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:05.148513 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02b907-fce8-4e01-a96f-e00197b866c3 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:05.153614 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:05.154229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:05.154509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:05.154780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] PATH_INFO: `/v3/users/f4e9ae1c32f44c12a040950121a38ab3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:05.155927 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e9ae1c32f44c12a040950121a38ab3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:05.157855 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:05.303830 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e9ae1c32f44c12a040950121a38ab3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:05.304397 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e9ae1c32f44c12a040950121a38ab3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:05.306065 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:05.381756 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e9ae1c32f44c12a040950121a38ab3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:05.394313 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e9ae1c32f44c12a040950121a38ab3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:05.404706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e9ae1c32f44c12a040950121a38ab3'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:05.408138 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1425/2846] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:43:05 2022] DELETE /identity/v3/users/f4e9ae1c32f44c12a040950121a38ab3 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:05.412652 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd36122-c11c-43bb-8892-52b9ab8e58c0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:05.413124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd36122-c11c-43bb-8892-52b9ab8e58c0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:05.413246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd36122-c11c-43bb-8892-52b9ab8e58c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:05.432336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-ccd36122-c11c-43bb-8892-52b9ab8e58c0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:05.433523 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ccd36122-c11c-43bb-8892-52b9ab8e58c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:05.491049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1422/2847] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:05.500867 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:05.502549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:05.561159 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274e9f3-73d8-4941-a19e-9a85b3d2a381 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:05.564725 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358c54e-a012-4282-abe0-6006f35d95c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:05.565328 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358c54e-a012-4282-abe0-6006f35d95c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:05.565606 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358c54e-a012-4282-abe0-6006f35d95c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:05.565879 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358c54e-a012-4282-abe0-6006f35d95c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:05.576160 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358c54e-a012-4282-abe0-6006f35d95c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:05.577302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358c54e-a012-4282-abe0-6006f35d95c9 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:05.586871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1426/2848] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:05.977846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd36122-c11c-43bb-8892-52b9ab8e58c0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:05.979264 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ccd36122-c11c-43bb-8892-52b9ab8e58c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:05.992480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd36122-c11c-43bb-8892-52b9ab8e58c0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:05.997024 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:05.997708 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:05.997985 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:05.998278 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] PATH_INFO: `/v3/projects/9d86903c504d4e388ee66df975eef99f` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:05.999409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d86903c504d4e388ee66df975eef99f)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:06.001602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:06.102020 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d86903c504d4e388ee66df975eef99f', 'request_id': 'req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:06.107432 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d86903c504d4e388ee66df975eef99f', 'request_id': 'req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:06.199063 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d86903c504d4e388ee66df975eef99f', 'request_id': 'req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:06.210803 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] The token cache is being invalidate because project 9d86903c504d4e388ee66df975eef99f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:43:06.211092 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:06.211769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:06.213287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1423/2849] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:43:05 2022] DELETE /identity/v3/projects/9d86903c504d4e388ee66df975eef99f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:06.217285 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358c54e-a012-4282-abe0-6006f35d95c9 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:06.227230 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358c54e-a012-4282-abe0-6006f35d95c9 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:06.282794 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:06.283584 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:06.283868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:06.284142 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] PATH_INFO: `/v3/users/e206ea197b364fcf86988d9ecd7e1396` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:06.285262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e206ea197b364fcf86988d9ecd7e1396)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:06.287221 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:06.509537 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e206ea197b364fcf86988d9ecd7e1396'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:06.510153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e206ea197b364fcf86988d9ecd7e1396'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:06.581414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e206ea197b364fcf86988d9ecd7e1396'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:06.593168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:06.594343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e206ea197b364fcf86988d9ecd7e1396'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:06.594985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-73c3b4be-dcb1-48f4-b37f-6f8b8b150935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e206ea197b364fcf86988d9ecd7e1396'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:06.607583 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1427/2850] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:43:06 2022] DELETE /identity/v3/users/e206ea197b364fcf86988d9ecd7e1396 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:07.057782 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:07.059246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:07.076250 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca87d63-c878-4b3c-833d-20a9a8d7eafc admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:07.090344 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:07.134733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:07.135950 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:07.136221 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:07.136487 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] PATH_INFO: `/v3/projects/82f9c0de5ce9435a92e4e59d88df138c` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:07.137592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f9c0de5ce9435a92e4e59d88df138c)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:07.139506 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:07.295176 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f9c0de5ce9435a92e4e59d88df138c', 'request_id': 'req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:07.300488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f9c0de5ce9435a92e4e59d88df138c', 'request_id': 'req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:07.381953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f9c0de5ce9435a92e4e59d88df138c', 'request_id': 'req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:07.394152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] The token cache is being invalidate because project 82f9c0de5ce9435a92e4e59d88df138c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:43:07.397444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:07.397444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-e71bc4aa-65dc-409e-af63-f274d4dcf1d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:07.397444 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1424/2851] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:43:07 2022] DELETE /identity/v3/projects/82f9c0de5ce9435a92e4e59d88df138c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:43:07.401495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145e16d-4a85-41df-9625-d84afe5d7a25 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:07.401998 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145e16d-4a85-41df-9625-d84afe5d7a25 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:07.402106 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145e16d-4a85-41df-9625-d84afe5d7a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:07.422579 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-f145e16d-4a85-41df-9625-d84afe5d7a25 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:07.426010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-f145e16d-4a85-41df-9625-d84afe5d7a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:07.482216 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1428/2852] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:07.490012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b60b5-a4e5-44ec-a33b-2c08da7d5724 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:07.490297 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b60b5-a4e5-44ec-a33b-2c08da7d5724 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:07.490572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b60b5-a4e5-44ec-a33b-2c08da7d5724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:07.512105 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c27b60b5-a4e5-44ec-a33b-2c08da7d5724 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:07.513291 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c27b60b5-a4e5-44ec-a33b-2c08da7d5724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:07.521114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1425/2853] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:07.526996 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145e16d-4a85-41df-9625-d84afe5d7a25 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:07.536636 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145e16d-4a85-41df-9625-d84afe5d7a25 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:07.541203 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc4e1e-571a-4ceb-9397-7cf7c3daab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:07.541789 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc4e1e-571a-4ceb-9397-7cf7c3daab6f admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:07.542069 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc4e1e-571a-4ceb-9397-7cf7c3daab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:07.542339 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc4e1e-571a-4ceb-9397-7cf7c3daab6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:07.542978 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc4e1e-571a-4ceb-9397-7cf7c3daab6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:43:07.543275 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc4e1e-571a-4ceb-9397-7cf7c3daab6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:07.544911 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc4e1e-571a-4ceb-9397-7cf7c3daab6f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:07.556514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1429/2854] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:43:07.571484 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b60b5-a4e5-44ec-a33b-2c08da7d5724 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:07.581286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b60b5-a4e5-44ec-a33b-2c08da7d5724 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:07.585963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062e33c-49b3-4972-b3a2-ff369ec26210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:07.586764 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062e33c-49b3-4972-b3a2-ff369ec26210 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:07.587255 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062e33c-49b3-4972-b3a2-ff369ec26210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:07.588287 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062e33c-49b3-4972-b3a2-ff369ec26210 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:07.588955 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062e33c-49b3-4972-b3a2-ff369ec26210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:07.590850 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062e33c-49b3-4972-b3a2-ff369ec26210 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:07.679816 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1426/2855] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:43:07 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:07.684420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc4e1e-571a-4ceb-9397-7cf7c3daab6f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:07.694376 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc4e1e-571a-4ceb-9397-7cf7c3daab6f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:07.750358 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61c6d8-4304-4d08-a4d9-fae9de8c4ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:07.750933 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61c6d8-4304-4d08-a4d9-fae9de8c4ede admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:07.751469 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61c6d8-4304-4d08-a4d9-fae9de8c4ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:07.751772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61c6d8-4304-4d08-a4d9-fae9de8c4ede admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:07.752395 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61c6d8-4304-4d08-a4d9-fae9de8c4ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:07.754297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61c6d8-4304-4d08-a4d9-fae9de8c4ede admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:07.861549 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1430/2856] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:43:07 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:43:07.865554 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062e33c-49b3-4972-b3a2-ff369ec26210 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:07.876883 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062e33c-49b3-4972-b3a2-ff369ec26210 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:07.881458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc2820-7503-4d4f-b927-0ebc968e67c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:07.882049 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc2820-7503-4d4f-b927-0ebc968e67c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:07.882323 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc2820-7503-4d4f-b927-0ebc968e67c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:07.882592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc2820-7503-4d4f-b927-0ebc968e67c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:07.883269 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc2820-7503-4d4f-b927-0ebc968e67c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:07.884888 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc2820-7503-4d4f-b927-0ebc968e67c0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:07.895353 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1427/2857] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:07.902459 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61c6d8-4304-4d08-a4d9-fae9de8c4ede admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:07.910199 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61c6d8-4304-4d08-a4d9-fae9de8c4ede admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:07.914843 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89b9a5-a647-4592-9a84-d53abb56ac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:07.915425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89b9a5-a647-4592-9a84-d53abb56ac71 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:07.915732 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89b9a5-a647-4592-9a84-d53abb56ac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:07.916003 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89b9a5-a647-4592-9a84-d53abb56ac71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:07.917201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89b9a5-a647-4592-9a84-d53abb56ac71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:07.918783 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89b9a5-a647-4592-9a84-d53abb56ac71 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:07.929141 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1431/2858] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:07.933870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc2820-7503-4d4f-b927-0ebc968e67c0 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:07.943742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc2820-7503-4d4f-b927-0ebc968e67c0 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:07.948517 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9e2ca-d491-4a3f-8f7b-662058ef2407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:07.949156 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9e2ca-d491-4a3f-8f7b-662058ef2407 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:07.949428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9e2ca-d491-4a3f-8f7b-662058ef2407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:07.949696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9e2ca-d491-4a3f-8f7b-662058ef2407 admin admin] PATH_INFO: `/v3/projects/b5ceb5ddf8dd4870889c39a7e32334f7/users/ba8cddf0cf7d451da6851a1607878a0a/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:07.968270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9e2ca-d491-4a3f-8f7b-662058ef2407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ceb5ddf8dd4870889c39a7e32334f7, user_id=ba8cddf0cf7d451da6851a1607878a0a, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:07.970862 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9e2ca-d491-4a3f-8f7b-662058ef2407 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:08.049864 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1428/2859] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:07 2022] PUT /identity/v3/projects/b5ceb5ddf8dd4870889c39a7e32334f7/users/ba8cddf0cf7d451da6851a1607878a0a/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:08.061989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f545c56-c3dc-421e-a281-c1d6de5cdeba None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:08.062265 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f545c56-c3dc-421e-a281-c1d6de5cdeba None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:08.062533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f545c56-c3dc-421e-a281-c1d6de5cdeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:08.090772 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-1f545c56-c3dc-421e-a281-c1d6de5cdeba None None] MFA Rules not processed for user `ba8cddf0cf7d451da6851a1607878a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:08.091982 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1f545c56-c3dc-421e-a281-c1d6de5cdeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:08.132987 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:08.155140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1432/2860] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:08.159614 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9e2ca-d491-4a3f-8f7b-662058ef2407 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:08.203989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:08.221037 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9e2ca-d491-4a3f-8f7b-662058ef2407 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:08.225730 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf8f92-3d37-4815-a655-76aca3c66881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:08.226569 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf8f92-3d37-4815-a655-76aca3c66881 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:08.227014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf8f92-3d37-4815-a655-76aca3c66881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:08.227683 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf8f92-3d37-4815-a655-76aca3c66881 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:08.228381 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf8f92-3d37-4815-a655-76aca3c66881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:08.230989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf8f92-3d37-4815-a655-76aca3c66881 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:08.297014 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1429/2861] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:43:08 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 12:43:08.301013 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f545c56-c3dc-421e-a281-c1d6de5cdeba None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:08.311942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f545c56-c3dc-421e-a281-c1d6de5cdeba None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:08.368808 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9fcb2-5865-459a-800b-e09e1d24a020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:08.369470 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9fcb2-5865-459a-800b-e09e1d24a020 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:08.369744 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9fcb2-5865-459a-800b-e09e1d24a020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:08.370011 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9fcb2-5865-459a-800b-e09e1d24a020 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:08.370619 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9fcb2-5865-459a-800b-e09e1d24a020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:08.372571 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9fcb2-5865-459a-800b-e09e1d24a020 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:08.478630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1433/2862] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:43:08 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:08.482697 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf8f92-3d37-4815-a655-76aca3c66881 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:08.493034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf8f92-3d37-4815-a655-76aca3c66881 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:08.497672 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef06d2-3871-48c8-a93f-0137b12ac644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:08.498267 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef06d2-3871-48c8-a93f-0137b12ac644 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:08.498543 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef06d2-3871-48c8-a93f-0137b12ac644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:08.498814 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef06d2-3871-48c8-a93f-0137b12ac644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:08.499447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef06d2-3871-48c8-a93f-0137b12ac644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:08.501045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef06d2-3871-48c8-a93f-0137b12ac644 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:08.511622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1430/2863] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:08.515444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9fcb2-5865-459a-800b-e09e1d24a020 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:08.525262 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9fcb2-5865-459a-800b-e09e1d24a020 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:08.530885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac8b0c-2a7f-44cf-b05a-356528e90db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:08.531473 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac8b0c-2a7f-44cf-b05a-356528e90db6 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:08.531778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac8b0c-2a7f-44cf-b05a-356528e90db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:08.532048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac8b0c-2a7f-44cf-b05a-356528e90db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:08.532712 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac8b0c-2a7f-44cf-b05a-356528e90db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:08.534345 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac8b0c-2a7f-44cf-b05a-356528e90db6 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:08.544289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1434/2864] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:08.550285 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef06d2-3871-48c8-a93f-0137b12ac644 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:08.560205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef06d2-3871-48c8-a93f-0137b12ac644 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:08.564949 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f527e-970a-4278-8b05-13efeb282113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:08.569111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f527e-970a-4278-8b05-13efeb282113 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:08.569111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f527e-970a-4278-8b05-13efeb282113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:08.569111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f527e-970a-4278-8b05-13efeb282113 admin admin] PATH_INFO: `/v3/projects/b03ea1ca20c643bbbb68ddb7cb8e7623/users/29f9c0ae976f48ef88180a2d48283b32/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:08.581707 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f527e-970a-4278-8b05-13efeb282113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03ea1ca20c643bbbb68ddb7cb8e7623, user_id=29f9c0ae976f48ef88180a2d48283b32, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:08.584373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f527e-970a-4278-8b05-13efeb282113 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:08.649555 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1431/2865] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:08 2022] PUT /identity/v3/projects/b03ea1ca20c643bbbb68ddb7cb8e7623/users/29f9c0ae976f48ef88180a2d48283b32/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:08.660437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f18fb2-c254-41dc-8627-23274db18997 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:08.660721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f18fb2-c254-41dc-8627-23274db18997 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:08.660999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f18fb2-c254-41dc-8627-23274db18997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:08.688121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-36f18fb2-c254-41dc-8627-23274db18997 None None] MFA Rules not processed for user `29f9c0ae976f48ef88180a2d48283b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:08.689336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-36f18fb2-c254-41dc-8627-23274db18997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:08.753233 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1435/2866] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:08.757625 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f527e-970a-4278-8b05-13efeb282113 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:08.820576 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f527e-970a-4278-8b05-13efeb282113 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:08.826111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8ddb0-20ff-4059-b077-e4d850515f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:08.826713 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8ddb0-20ff-4059-b077-e4d850515f11 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:08.826995 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8ddb0-20ff-4059-b077-e4d850515f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:08.827270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8ddb0-20ff-4059-b077-e4d850515f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:08.827947 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8ddb0-20ff-4059-b077-e4d850515f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:08.829876 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8ddb0-20ff-4059-b077-e4d850515f11 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:08.909229 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1432/2867] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:43:08 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:08.913448 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f18fb2-c254-41dc-8627-23274db18997 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:08.924941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f18fb2-c254-41dc-8627-23274db18997 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:08.971840 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:08.982086 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46784819-f7b9-42b3-9cc8-fcd3b340110b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:08.982849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46784819-f7b9-42b3-9cc8-fcd3b340110b admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:08.983132 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46784819-f7b9-42b3-9cc8-fcd3b340110b admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:08.983429 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46784819-f7b9-42b3-9cc8-fcd3b340110b admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:08.984111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46784819-f7b9-42b3-9cc8-fcd3b340110b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:08.986175 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46784819-f7b9-42b3-9cc8-fcd3b340110b admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:09.061856 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1436/2868] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:43:08 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:09.066303 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8ddb0-20ff-4059-b077-e4d850515f11 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:09.076804 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8ddb0-20ff-4059-b077-e4d850515f11 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:09.081591 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcf045-30da-4195-b4bd-2dd3421e1665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:09.082184 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcf045-30da-4195-b4bd-2dd3421e1665 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:09.082499 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcf045-30da-4195-b4bd-2dd3421e1665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:09.082775 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcf045-30da-4195-b4bd-2dd3421e1665 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:09.083409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcf045-30da-4195-b4bd-2dd3421e1665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:09.085620 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcf045-30da-4195-b4bd-2dd3421e1665 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:09.100068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1433/2869] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 12:43:09.101300 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46784819-f7b9-42b3-9cc8-fcd3b340110b admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:09.112404 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46784819-f7b9-42b3-9cc8-fcd3b340110b admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:09.117107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33984b-d122-41a7-8179-48acf2d0b1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:09.117755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33984b-d122-41a7-8179-48acf2d0b1d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:09.118031 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33984b-d122-41a7-8179-48acf2d0b1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:09.118302 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33984b-d122-41a7-8179-48acf2d0b1d8 admin admin] PATH_INFO: `/v3/projects/c51946e1c20c49c29a916baea34d6417/users/128d9d440c864b8892ba14f373f4c730/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:09.133639 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33984b-d122-41a7-8179-48acf2d0b1d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c51946e1c20c49c29a916baea34d6417, user_id=128d9d440c864b8892ba14f373f4c730, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:09.136452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33984b-d122-41a7-8179-48acf2d0b1d8 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:09.216210 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1437/2870] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:09 2022] PUT /identity/v3/projects/c51946e1c20c49c29a916baea34d6417/users/128d9d440c864b8892ba14f373f4c730/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:43:09.220173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcf045-30da-4195-b4bd-2dd3421e1665 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:09.230098 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcf045-30da-4195-b4bd-2dd3421e1665 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:09.285990 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a1f1-8a62-4bd5-92b2-5c80a92a62c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:09.286623 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a1f1-8a62-4bd5-92b2-5c80a92a62c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:09.286899 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a1f1-8a62-4bd5-92b2-5c80a92a62c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:09.287173 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a1f1-8a62-4bd5-92b2-5c80a92a62c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:09.287839 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2a1f1-8a62-4bd5-92b2-5c80a92a62c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:09.289425 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2a1f1-8a62-4bd5-92b2-5c80a92a62c0 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:09.299742 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1434/2871] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:09.303778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33984b-d122-41a7-8179-48acf2d0b1d8 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:09.313890 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33984b-d122-41a7-8179-48acf2d0b1d8 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:09.318455 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd1f69-a85a-4b72-a0ea-249f7358df28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:09.319089 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd1f69-a85a-4b72-a0ea-249f7358df28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:09.319368 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd1f69-a85a-4b72-a0ea-249f7358df28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:09.319672 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd1f69-a85a-4b72-a0ea-249f7358df28 admin admin] PATH_INFO: `/v3/projects/c51946e1c20c49c29a916baea34d6417/users/128d9d440c864b8892ba14f373f4c730/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:09.321562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd1f69-a85a-4b72-a0ea-249f7358df28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c51946e1c20c49c29a916baea34d6417, user_id=128d9d440c864b8892ba14f373f4c730, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:09.324682 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd1f69-a85a-4b72-a0ea-249f7358df28 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:09.334498 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1438/2872] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:09 2022] PUT /identity/v3/projects/c51946e1c20c49c29a916baea34d6417/users/128d9d440c864b8892ba14f373f4c730/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:09.344776 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8794d-0eb2-4081-b550-a20afa8e7dc4 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:09.345071 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8794d-0eb2-4081-b550-a20afa8e7dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:09.345343 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8794d-0eb2-4081-b550-a20afa8e7dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:09.352376 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:09.372778 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-37e8794d-0eb2-4081-b550-a20afa8e7dc4 None None] MFA Rules not processed for user `128d9d440c864b8892ba14f373f4c730`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:09.373978 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-37e8794d-0eb2-4081-b550-a20afa8e7dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:09.442671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1435/2873] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:09.454847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd1f69-a85a-4b72-a0ea-249f7358df28 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:09.456596 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-64cd1f69-a85a-4b72-a0ea-249f7358df28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:09.513537 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd1f69-a85a-4b72-a0ea-249f7358df28 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:09.517279 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27954e-8973-4652-ad2c-9eeb6f5ea728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:09.517871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27954e-8973-4652-ad2c-9eeb6f5ea728 service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:09.518148 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27954e-8973-4652-ad2c-9eeb6f5ea728 service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:09.518420 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27954e-8973-4652-ad2c-9eeb6f5ea728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:09.576701 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27954e-8973-4652-ad2c-9eeb6f5ea728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:09.578104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27954e-8973-4652-ad2c-9eeb6f5ea728 service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:09.586935 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1439/2874] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:11.465702 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8794d-0eb2-4081-b550-a20afa8e7dc4 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:11.467143 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-37e8794d-0eb2-4081-b550-a20afa8e7dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:11.479705 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8794d-0eb2-4081-b550-a20afa8e7dc4 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:11.483270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11491e4a-2c24-493f-a5c7-449474bfb0c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:11.483905 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11491e4a-2c24-493f-a5c7-449474bfb0c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:11.484187 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11491e4a-2c24-493f-a5c7-449474bfb0c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:11.484480 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11491e4a-2c24-493f-a5c7-449474bfb0c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:11.540458 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11491e4a-2c24-493f-a5c7-449474bfb0c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:11.542395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11491e4a-2c24-493f-a5c7-449474bfb0c2 service cinder] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:11.551093 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1436/2875] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:13.096472 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27954e-8973-4652-ad2c-9eeb6f5ea728 service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:13.105777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27954e-8973-4652-ad2c-9eeb6f5ea728 service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:13.109375 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cc8d7-a8d0-48ba-ba3b-d6bdaa3fde8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:13.109965 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cc8d7-a8d0-48ba-ba3b-d6bdaa3fde8f service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:13.110244 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cc8d7-a8d0-48ba-ba3b-d6bdaa3fde8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:13.110515 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cc8d7-a8d0-48ba-ba3b-d6bdaa3fde8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:13.119307 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cc8d7-a8d0-48ba-ba3b-d6bdaa3fde8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:13.121421 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cc8d7-a8d0-48ba-ba3b-d6bdaa3fde8f service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:13.130922 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1440/2876] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:17.790545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11491e4a-2c24-493f-a5c7-449474bfb0c2 service cinder] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:17.852844 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11491e4a-2c24-493f-a5c7-449474bfb0c2 service cinder] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:17.857336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:17.858566 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:17.858845 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:17.859117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] PATH_INFO: `/v3/users/ba8cddf0cf7d451da6851a1607878a0a` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:17.860332 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba8cddf0cf7d451da6851a1607878a0a)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:17.862233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:17.989065 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:18.091733 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba8cddf0cf7d451da6851a1607878a0a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:18.092953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba8cddf0cf7d451da6851a1607878a0a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:18.175354 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba8cddf0cf7d451da6851a1607878a0a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:18.187919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba8cddf0cf7d451da6851a1607878a0a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:18.188556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba8cddf0cf7d451da6851a1607878a0a'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:18.200666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1437/2877] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:43:17 2022] DELETE /identity/v3/users/ba8cddf0cf7d451da6851a1607878a0a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:18.205623 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df0d58-563b-4595-b35e-c8a163e4e09a None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:18.206090 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df0d58-563b-4595-b35e-c8a163e4e09a None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:18.206211 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df0d58-563b-4595-b35e-c8a163e4e09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:18.225481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-70df0d58-563b-4595-b35e-c8a163e4e09a None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:18.226677 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-70df0d58-563b-4595-b35e-c8a163e4e09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:18.265537 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:18.284514 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1441/2878] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:18.294342 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:18.295767 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:18.354017 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a12e3a-1ca5-425c-b2de-b274229e326d admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:18.357631 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486618ce-df11-4d64-8121-c327c127df87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:18.358236 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486618ce-df11-4d64-8121-c327c127df87 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:18.358511 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486618ce-df11-4d64-8121-c327c127df87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:18.358780 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486618ce-df11-4d64-8121-c327c127df87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:18.367592 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486618ce-df11-4d64-8121-c327c127df87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:18.369010 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486618ce-df11-4d64-8121-c327c127df87 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:18.378582 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1438/2879] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:18.828959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df0d58-563b-4595-b35e-c8a163e4e09a None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:18.830604 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-70df0d58-563b-4595-b35e-c8a163e4e09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:18.845018 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df0d58-563b-4595-b35e-c8a163e4e09a None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:18.849823 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:18.851597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:18.851597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:18.851597 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] PATH_INFO: `/v3/projects/b5ceb5ddf8dd4870889c39a7e32334f7` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:18.852938 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ceb5ddf8dd4870889c39a7e32334f7)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:18.855272 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:19.000218 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ceb5ddf8dd4870889c39a7e32334f7', 'request_id': 'req-ea28779f-83e8-42aa-a695-e3cd49fde6ba'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:19.006289 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ceb5ddf8dd4870889c39a7e32334f7', 'request_id': 'req-ea28779f-83e8-42aa-a695-e3cd49fde6ba'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:19.095561 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ceb5ddf8dd4870889c39a7e32334f7', 'request_id': 'req-ea28779f-83e8-42aa-a695-e3cd49fde6ba'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:19.107776 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] The token cache is being invalidate because project b5ceb5ddf8dd4870889c39a7e32334f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:43:19.108063 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:19.109177 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:19.110649 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1442/2880] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:43:18 2022] DELETE /identity/v3/projects/b5ceb5ddf8dd4870889c39a7e32334f7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:43:19.114736 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486618ce-df11-4d64-8121-c327c127df87 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:19.125012 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486618ce-df11-4d64-8121-c327c127df87 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:19.180772 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:19.181535 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:19.181821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:19.182146 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] PATH_INFO: `/v3/users/29f9c0ae976f48ef88180a2d48283b32` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:19.183203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29f9c0ae976f48ef88180a2d48283b32)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:19.185155 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:19.434874 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29f9c0ae976f48ef88180a2d48283b32'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:19.435459 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29f9c0ae976f48ef88180a2d48283b32'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:19.438003 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:19.524324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29f9c0ae976f48ef88180a2d48283b32'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:19.537038 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29f9c0ae976f48ef88180a2d48283b32'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:19.537709 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29f9c0ae976f48ef88180a2d48283b32'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:19.550258 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1439/2881] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:43:19 2022] DELETE /identity/v3/users/29f9c0ae976f48ef88180a2d48283b32 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:20.011895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:20.013353 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:20.027574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28779f-83e8-42aa-a695-e3cd49fde6ba admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:20.097041 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:20.097680 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:20.097960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:20.098234 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] PATH_INFO: `/v3/projects/b03ea1ca20c643bbbb68ddb7cb8e7623` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:20.099342 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b03ea1ca20c643bbbb68ddb7cb8e7623)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:20.101336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:20.216456 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b03ea1ca20c643bbbb68ddb7cb8e7623', 'request_id': 'req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:20.221923 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b03ea1ca20c643bbbb68ddb7cb8e7623', 'request_id': 'req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:20.301271 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b03ea1ca20c643bbbb68ddb7cb8e7623', 'request_id': 'req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:20.308667 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:20.313643 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] The token cache is being invalidate because project b03ea1ca20c643bbbb68ddb7cb8e7623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:43:20.313929 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:20.314542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:20.316056 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1443/2882] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:43:20 2022] DELETE /identity/v3/projects/b03ea1ca20c643bbbb68ddb7cb8e7623 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:20.320398 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:20.328608 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:20.335495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd5ce7-086c-4f62-a20e-5b4cef4d4317 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:20.392969 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:20.393610 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:20.393887 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:20.394159 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] PATH_INFO: `/v3/users/128d9d440c864b8892ba14f373f4c730` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:20.395277 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=128d9d440c864b8892ba14f373f4c730)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:20.397222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:20.588752 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128d9d440c864b8892ba14f373f4c730'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:20.589324 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128d9d440c864b8892ba14f373f4c730'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:20.678406 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128d9d440c864b8892ba14f373f4c730'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:20.694572 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128d9d440c864b8892ba14f373f4c730'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:20.695208 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-d171fb2a-c1e8-4369-94e5-9d1ff1292877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128d9d440c864b8892ba14f373f4c730'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:20.708877 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1440/2883] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:43:20 2022] DELETE /identity/v3/users/128d9d440c864b8892ba14f373f4c730 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:21.160576 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:21.166522 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:21.175425 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c3d89-ebda-4bff-b626-fbbd03dd55dc admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:21.232832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:21.233503 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:21.233780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:21.234057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] PATH_INFO: `/v3/projects/c51946e1c20c49c29a916baea34d6417` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:21.235168 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c51946e1c20c49c29a916baea34d6417)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:21.237095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:21.358983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c51946e1c20c49c29a916baea34d6417', 'request_id': 'req-9f1e123f-dae1-4ca2-84c1-37ac519cce02'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:21.364446 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c51946e1c20c49c29a916baea34d6417', 'request_id': 'req-9f1e123f-dae1-4ca2-84c1-37ac519cce02'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:21.432851 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c51946e1c20c49c29a916baea34d6417', 'request_id': 'req-9f1e123f-dae1-4ca2-84c1-37ac519cce02'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:21.444618 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] The token cache is being invalidate because project c51946e1c20c49c29a916baea34d6417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81648) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:43:21.444913 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:21.445572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-9f1e123f-dae1-4ca2-84c1-37ac519cce02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:21.447048 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1444/2884] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:43:21 2022] DELETE /identity/v3/projects/c51946e1c20c49c29a916baea34d6417 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:21.451989 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f1e3b-711b-4967-a0a9-38bc75ac0ea0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:21.453222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f1e3b-711b-4967-a0a9-38bc75ac0ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:21.453222 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f1e3b-711b-4967-a0a9-38bc75ac0ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:21.472500 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-a83f1e3b-711b-4967-a0a9-38bc75ac0ea0 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:21.473692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-a83f1e3b-711b-4967-a0a9-38bc75ac0ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:21.515908 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:21.532792 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1441/2885] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:21.540495 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43ca62-00a9-4024-a20f-1f271cc7d838 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:21.540777 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43ca62-00a9-4024-a20f-1f271cc7d838 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:21.541049 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43ca62-00a9-4024-a20f-1f271cc7d838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:21.560666 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-3f43ca62-00a9-4024-a20f-1f271cc7d838 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:21.562523 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-3f43ca62-00a9-4024-a20f-1f271cc7d838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:21.570447 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1445/2886] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:21.580890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f1e3b-711b-4967-a0a9-38bc75ac0ea0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:21.590665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f1e3b-711b-4967-a0a9-38bc75ac0ea0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:21.595526 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c725d2-8a52-4ea0-9acc-c05e83292cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:21.596303 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c725d2-8a52-4ea0-9acc-c05e83292cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:21.596665 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c725d2-8a52-4ea0-9acc-c05e83292cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:21.596941 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c725d2-8a52-4ea0-9acc-c05e83292cf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:21.597602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c725d2-8a52-4ea0-9acc-c05e83292cf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81649) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:43:21.597910 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c725d2-8a52-4ea0-9acc-c05e83292cf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:21.599505 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c725d2-8a52-4ea0-9acc-c05e83292cf9 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:21.611829 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1442/2887] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:43:21.616227 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43ca62-00a9-4024-a20f-1f271cc7d838 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:21.626408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43ca62-00a9-4024-a20f-1f271cc7d838 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:21.631120 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50718699-cff6-4ab7-aa90-b20b102af024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:21.631758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50718699-cff6-4ab7-aa90-b20b102af024 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:21.632033 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50718699-cff6-4ab7-aa90-b20b102af024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:21.632331 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50718699-cff6-4ab7-aa90-b20b102af024 admin admin] PATH_INFO: `/v3/projects` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:21.632963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50718699-cff6-4ab7-aa90-b20b102af024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:21.636539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50718699-cff6-4ab7-aa90-b20b102af024 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:21.755281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1446/2888] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:43:21 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:21.760663 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c725d2-8a52-4ea0-9acc-c05e83292cf9 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:21.771465 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c725d2-8a52-4ea0-9acc-c05e83292cf9 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:21.827036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f1037-87fc-481d-8163-b91c90ebda80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:21.828520 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f1037-87fc-481d-8163-b91c90ebda80 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:21.828799 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f1037-87fc-481d-8163-b91c90ebda80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:21.829068 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f1037-87fc-481d-8163-b91c90ebda80 admin admin] PATH_INFO: `/v3/users` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:21.829704 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f1037-87fc-481d-8163-b91c90ebda80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:21.831724 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f1037-87fc-481d-8163-b91c90ebda80 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:21.981242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1443/2889] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:43:21 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:21.985865 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50718699-cff6-4ab7-aa90-b20b102af024 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:21.996413 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50718699-cff6-4ab7-aa90-b20b102af024 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:22.001868 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f9391-8b30-4bf0-aeb4-ebad3658005d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:22.002509 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f9391-8b30-4bf0-aeb4-ebad3658005d admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:22.002787 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f9391-8b30-4bf0-aeb4-ebad3658005d admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:22.003059 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f9391-8b30-4bf0-aeb4-ebad3658005d admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:22.003755 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f9391-8b30-4bf0-aeb4-ebad3658005d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:22.005417 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f9391-8b30-4bf0-aeb4-ebad3658005d admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:22.015281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1447/2890] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:22.019101 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f1037-87fc-481d-8163-b91c90ebda80 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:22.029312 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f1037-87fc-481d-8163-b91c90ebda80 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:22.034089 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908296c-61be-421c-bf6a-559f7f58611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:22.034679 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908296c-61be-421c-bf6a-559f7f58611f admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:22.034960 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908296c-61be-421c-bf6a-559f7f58611f admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:22.035235 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908296c-61be-421c-bf6a-559f7f58611f admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:22.035912 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908296c-61be-421c-bf6a-559f7f58611f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:22.037486 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908296c-61be-421c-bf6a-559f7f58611f admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:22.049904 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1444/2891] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:22.054249 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f9391-8b30-4bf0-aeb4-ebad3658005d admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:22.064963 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f9391-8b30-4bf0-aeb4-ebad3658005d admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:22.069539 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55353d2-9e17-4646-8267-78febf60d36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:22.070180 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55353d2-9e17-4646-8267-78febf60d36f admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:22.070453 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55353d2-9e17-4646-8267-78febf60d36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:22.070721 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55353d2-9e17-4646-8267-78febf60d36f admin admin] PATH_INFO: `/v3/projects/e07c3243b46848009fe81a038b42b882/users/7e60b260fb664ce8ae2ad8bb65c155c0/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:22.085270 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55353d2-9e17-4646-8267-78febf60d36f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07c3243b46848009fe81a038b42b882, user_id=7e60b260fb664ce8ae2ad8bb65c155c0, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:22.087937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55353d2-9e17-4646-8267-78febf60d36f admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:22.193762 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1448/2892] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:22 2022] PUT /identity/v3/projects/e07c3243b46848009fe81a038b42b882/users/7e60b260fb664ce8ae2ad8bb65c155c0/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:22.204119 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd9047-a28f-41c9-bce5-907eafff95d0 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:22.204398 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd9047-a28f-41c9-bce5-907eafff95d0 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:22.204669 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd9047-a28f-41c9-bce5-907eafff95d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:22.232041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-39cd9047-a28f-41c9-bce5-907eafff95d0 None None] MFA Rules not processed for user `7e60b260fb664ce8ae2ad8bb65c155c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:22.233231 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-39cd9047-a28f-41c9-bce5-907eafff95d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:22.297617 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1445/2893] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:22.308015 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55353d2-9e17-4646-8267-78febf60d36f admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:22.309730 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c55353d2-9e17-4646-8267-78febf60d36f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:22.336832 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:22.365918 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55353d2-9e17-4646-8267-78febf60d36f admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:22.369847 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aaf9f-674d-41d5-bdf9-5f3f98ad05bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f33f1f3d1ca4436887c2346bd9fa149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:22.370452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aaf9f-674d-41d5-bdf9-5f3f98ad05bc service cinder] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:22.370904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aaf9f-674d-41d5-bdf9-5f3f98ad05bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:22.371565 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aaf9f-674d-41d5-bdf9-5f3f98ad05bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:22.380050 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aaf9f-674d-41d5-bdf9-5f3f98ad05bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:22.383193 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aaf9f-674d-41d5-bdf9-5f3f98ad05bc service cinder] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:22.390101 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1449/2894] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:30.527833 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd9047-a28f-41c9-bce5-907eafff95d0 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:30.529387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-39cd9047-a28f-41c9-bce5-907eafff95d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:30.574907 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:30.599813 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd9047-a28f-41c9-bce5-907eafff95d0 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:30.604901 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf8cd6-96c4-45e8-bb3f-a945425f6285 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:30.605498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf8cd6-96c4-45e8-bb3f-a945425f6285 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:30.605765 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf8cd6-96c4-45e8-bb3f-a945425f6285 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:30.606035 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf8cd6-96c4-45e8-bb3f-a945425f6285 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:30.607970 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-9aaf8cd6-96c4-45e8-bb3f-a945425f6285 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:30.620274 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf8cd6-96c4-45e8-bb3f-a945425f6285 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:30.621919 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf8cd6-96c4-45e8-bb3f-a945425f6285 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:30.633641 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1446/2895] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:32.461105 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=15 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:34.282147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aaf9f-674d-41d5-bdf9-5f3f98ad05bc service cinder] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:34.342121 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aaf9f-674d-41d5-bdf9-5f3f98ad05bc service cinder] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:34.346598 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:34.347206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:34.347481 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:34.347778 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] PATH_INFO: `/v3/users/7e60b260fb664ce8ae2ad8bb65c155c0` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:34.348891 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e60b260fb664ce8ae2ad8bb65c155c0)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:34.350796 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:34.566281 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e60b260fb664ce8ae2ad8bb65c155c0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:34.566999 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e60b260fb664ce8ae2ad8bb65c155c0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:34.655080 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e60b260fb664ce8ae2ad8bb65c155c0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:34.667914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e60b260fb664ce8ae2ad8bb65c155c0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:34.668664 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e60b260fb664ce8ae2ad8bb65c155c0'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:34.681280 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1450/2896] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:43:34 2022] DELETE /identity/v3/users/7e60b260fb664ce8ae2ad8bb65c155c0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:34.685920 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169bb8c-8b0a-495f-ae9d-16552110791e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:34.686420 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169bb8c-8b0a-495f-ae9d-16552110791e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:34.686628 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169bb8c-8b0a-495f-ae9d-16552110791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:34.705205 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f169bb8c-8b0a-495f-ae9d-16552110791e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:34.706395 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f169bb8c-8b0a-495f-ae9d-16552110791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:34.764842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1447/2897] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:34.774626 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:34.777644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:34.834525 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697a92b-bae3-4181-a515-a774cdafefaf admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:34.838336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8206a-b43f-4c4a-947b-96b55cbe4008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:34.838937 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8206a-b43f-4c4a-947b-96b55cbe4008 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:34.839219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8206a-b43f-4c4a-947b-96b55cbe4008 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:34.839493 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8206a-b43f-4c4a-947b-96b55cbe4008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:34.848679 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8206a-b43f-4c4a-947b-96b55cbe4008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:34.850818 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8206a-b43f-4c4a-947b-96b55cbe4008 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:34.860482 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1451/2898] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:35.348361 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169bb8c-8b0a-495f-ae9d-16552110791e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:35.349854 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f169bb8c-8b0a-495f-ae9d-16552110791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:35.362281 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169bb8c-8b0a-495f-ae9d-16552110791e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:35.367009 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:35.367660 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:35.367933 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:35.368202 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] PATH_INFO: `/v3/projects/e07c3243b46848009fe81a038b42b882` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:35.369357 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07c3243b46848009fe81a038b42b882)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:35.371263 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:35.536207 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07c3243b46848009fe81a038b42b882', 'request_id': 'req-a04ab3ee-200e-4e52-8699-e2e27264e034'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:35.541956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07c3243b46848009fe81a038b42b882', 'request_id': 'req-a04ab3ee-200e-4e52-8699-e2e27264e034'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:35.661158 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07c3243b46848009fe81a038b42b882', 'request_id': 'req-a04ab3ee-200e-4e52-8699-e2e27264e034'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:35.672958 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] The token cache is being invalidate because project e07c3243b46848009fe81a038b42b882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:43:35.673249 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:35.673886 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-a04ab3ee-200e-4e52-8699-e2e27264e034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:43:35.675370 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1448/2899] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:43:35 2022] DELETE /identity/v3/projects/e07c3243b46848009fe81a038b42b882 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:35.680871 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35789b3-7737-4235-8ed3-6af9adadcc6e None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:35.681374 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35789b3-7737-4235-8ed3-6af9adadcc6e None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:35.681490 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35789b3-7737-4235-8ed3-6af9adadcc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:35.701363 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-a35789b3-7737-4235-8ed3-6af9adadcc6e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:35.702542 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-a35789b3-7737-4235-8ed3-6af9adadcc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:35.718959 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:35.761813 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1452/2900] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:35.770809 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98591-8ebf-452a-8294-869a40e7427e None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:35.775377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98591-8ebf-452a-8294-869a40e7427e None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:35.775377 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98591-8ebf-452a-8294-869a40e7427e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:35.791715 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-f9e98591-8ebf-452a-8294-869a40e7427e None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:35.792928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f9e98591-8ebf-452a-8294-869a40e7427e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:35.800818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1449/2901] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:35.805511 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35789b3-7737-4235-8ed3-6af9adadcc6e None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:35.815215 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35789b3-7737-4235-8ed3-6af9adadcc6e None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:35.820634 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5d9a1-6beb-4963-8e14-696b20a6d916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:35.821431 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5d9a1-6beb-4963-8e14-696b20a6d916 admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:35.821719 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5d9a1-6beb-4963-8e14-696b20a6d916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:35.822295 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5d9a1-6beb-4963-8e14-696b20a6d916 admin admin] PATH_INFO: `/v3/domains` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:35.822932 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5d9a1-6beb-4963-8e14-696b20a6d916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81648) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 12:43:35.823232 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5d9a1-6beb-4963-8e14-696b20a6d916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:35.824866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5d9a1-6beb-4963-8e14-696b20a6d916 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:35.836519 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1453/2902] 10.222.0.62 () {58 vars in 1153 bytes} [Mon Sep 5 12:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 12:43:35.840684 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e98591-8ebf-452a-8294-869a40e7427e None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:35.849858 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e98591-8ebf-452a-8294-869a40e7427e None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:35.854527 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf24c7-ef67-42aa-a2f5-d3e1191eb506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:35.855113 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf24c7-ef67-42aa-a2f5-d3e1191eb506 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:35.855389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf24c7-ef67-42aa-a2f5-d3e1191eb506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:35.855710 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf24c7-ef67-42aa-a2f5-d3e1191eb506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:35.856336 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf24c7-ef67-42aa-a2f5-d3e1191eb506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:35.858218 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf24c7-ef67-42aa-a2f5-d3e1191eb506 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:35.953567 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1450/2903] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:43:35 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:35.957904 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5d9a1-6beb-4963-8e14-696b20a6d916 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:35.968022 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5d9a1-6beb-4963-8e14-696b20a6d916 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:36.025408 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edb203-9805-4183-a4d3-dae7d74efe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:36.026016 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edb203-9805-4183-a4d3-dae7d74efe68 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:36.026325 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edb203-9805-4183-a4d3-dae7d74efe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:36.026600 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edb203-9805-4183-a4d3-dae7d74efe68 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:36.027220 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edb203-9805-4183-a4d3-dae7d74efe68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:36.029248 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edb203-9805-4183-a4d3-dae7d74efe68 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:36.141308 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1454/2904] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:43:35 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:36.145429 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf24c7-ef67-42aa-a2f5-d3e1191eb506 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:36.155821 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf24c7-ef67-42aa-a2f5-d3e1191eb506 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:36.160528 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf86a0-5e1b-4b49-b041-ae770a7bc024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:36.161124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf86a0-5e1b-4b49-b041-ae770a7bc024 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:36.161400 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf86a0-5e1b-4b49-b041-ae770a7bc024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:36.161673 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf86a0-5e1b-4b49-b041-ae770a7bc024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:36.162837 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf86a0-5e1b-4b49-b041-ae770a7bc024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:36.164456 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf86a0-5e1b-4b49-b041-ae770a7bc024 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:36.174296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1451/2905] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:36.178797 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edb203-9805-4183-a4d3-dae7d74efe68 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:36.189115 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edb203-9805-4183-a4d3-dae7d74efe68 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:36.193816 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b5e0e-7a57-4830-ac09-738368549c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:36.194407 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b5e0e-7a57-4830-ac09-738368549c3c admin admin] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:36.194686 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b5e0e-7a57-4830-ac09-738368549c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:36.194960 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b5e0e-7a57-4830-ac09-738368549c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:36.195646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b5e0e-7a57-4830-ac09-738368549c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:36.197255 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b5e0e-7a57-4830-ac09-738368549c3c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:36.207970 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1455/2906] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:36.212025 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf86a0-5e1b-4b49-b041-ae770a7bc024 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:36.221884 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf86a0-5e1b-4b49-b041-ae770a7bc024 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:36.226963 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0b1e3-880d-4a5c-9e2f-b6058520e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:36.227642 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0b1e3-880d-4a5c-9e2f-b6058520e5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:36.227922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0b1e3-880d-4a5c-9e2f-b6058520e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:36.228193 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0b1e3-880d-4a5c-9e2f-b6058520e5a1 admin admin] PATH_INFO: `/v3/projects/58a867fbd2d444728391be6de51146c9/users/e4315027930746c2b037efa139531914/roles/0062155b0fbf41a19c9110ad71a046e3` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:36.239794 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:36.243447 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0b1e3-880d-4a5c-9e2f-b6058520e5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a867fbd2d444728391be6de51146c9, user_id=e4315027930746c2b037efa139531914, role_id=0062155b0fbf41a19c9110ad71a046e3)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:36.246117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0b1e3-880d-4a5c-9e2f-b6058520e5a1 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:36.374498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1452/2907] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:36 2022] PUT /identity/v3/projects/58a867fbd2d444728391be6de51146c9/users/e4315027930746c2b037efa139531914/roles/0062155b0fbf41a19c9110ad71a046e3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:36.379941 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc4215-cf88-4070-9338-11a1af785d56 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:36.380229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc4215-cf88-4070-9338-11a1af785d56 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:36.380533 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc4215-cf88-4070-9338-11a1af785d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:36.399638 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-41cc4215-cf88-4070-9338-11a1af785d56 None None] MFA Rules not processed for user `57929111b78840da8ec77704273d56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:36.400835 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-41cc4215-cf88-4070-9338-11a1af785d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:36.461097 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1456/2908] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:36.471386 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0b1e3-880d-4a5c-9e2f-b6058520e5a1 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:36.473121 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f4b0b1e3-880d-4a5c-9e2f-b6058520e5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:36.531587 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0b1e3-880d-4a5c-9e2f-b6058520e5a1 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:36.535246 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556a924-d276-4d27-8b0e-8e420f57420f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:36.535922 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a924-d276-4d27-8b0e-8e420f57420f service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:36.536203 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a924-d276-4d27-8b0e-8e420f57420f service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:36.536503 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a924-d276-4d27-8b0e-8e420f57420f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:36.546296 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556a924-d276-4d27-8b0e-8e420f57420f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:36.547842 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556a924-d276-4d27-8b0e-8e420f57420f service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:36.558016 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1453/2909] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:42.119630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e19118-87e3-42ed-8e47-bf24b4fb6747 None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:42.119630 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e19118-87e3-42ed-8e47-bf24b4fb6747 None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:42.120297 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e19118-87e3-42ed-8e47-bf24b4fb6747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:42.152311 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-67e19118-87e3-42ed-8e47-bf24b4fb6747 None None] MFA Rules not processed for user `e4315027930746c2b037efa139531914`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:42.153616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-67e19118-87e3-42ed-8e47-bf24b4fb6747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:42.173553 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:42.232107 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1457/2910] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:42.236137 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556a924-d276-4d27-8b0e-8e420f57420f service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:42.247750 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556a924-d276-4d27-8b0e-8e420f57420f service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:42.252786 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8226bf-4b7e-4df1-8030-5e69dd805333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:42.253387 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8226bf-4b7e-4df1-8030-5e69dd805333 admin admin] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:42.253658 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8226bf-4b7e-4df1-8030-5e69dd805333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:42.254153 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8226bf-4b7e-4df1-8030-5e69dd805333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:42.254918 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8226bf-4b7e-4df1-8030-5e69dd805333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:42.256600 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8226bf-4b7e-4df1-8030-5e69dd805333 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:42.409033 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1454/2911] 10.222.0.62 () {60 vars in 1153 bytes} [Mon Sep 5 12:43:42 2022] POST /identity/v3/projects => generated 371 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:42.411645 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e19118-87e3-42ed-8e47-bf24b4fb6747 None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:42.424076 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e19118-87e3-42ed-8e47-bf24b4fb6747 None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:42.485371 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8c030-f525-4a33-a5f0-ccd556e997a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:42.485985 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8c030-f525-4a33-a5f0-ccd556e997a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:42.486277 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8c030-f525-4a33-a5f0-ccd556e997a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:42.486551 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8c030-f525-4a33-a5f0-ccd556e997a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:42.487179 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8c030-f525-4a33-a5f0-ccd556e997a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:42.489140 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8c030-f525-4a33-a5f0-ccd556e997a2 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:42.626714 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1458/2912] 10.222.0.62 () {60 vars in 1144 bytes} [Mon Sep 5 12:43:42 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 12:43:42.630870 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8226bf-4b7e-4df1-8030-5e69dd805333 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:42.641961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8226bf-4b7e-4df1-8030-5e69dd805333 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:42.646556 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab68a7f-1d5d-4004-be7d-9cfea18940cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:42.647151 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab68a7f-1d5d-4004-be7d-9cfea18940cf admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:42.648045 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab68a7f-1d5d-4004-be7d-9cfea18940cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:42.648322 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab68a7f-1d5d-4004-be7d-9cfea18940cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:42.648959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab68a7f-1d5d-4004-be7d-9cfea18940cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:42.650635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab68a7f-1d5d-4004-be7d-9cfea18940cf admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:42.661449 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1455/2913] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:42.665465 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8c030-f525-4a33-a5f0-ccd556e997a2 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:42.676508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8c030-f525-4a33-a5f0-ccd556e997a2 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:42.681196 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b76614-91cc-4534-b7ca-14d843a87118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:42.681917 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b76614-91cc-4534-b7ca-14d843a87118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:42.682201 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b76614-91cc-4534-b7ca-14d843a87118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:42.682474 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b76614-91cc-4534-b7ca-14d843a87118 admin admin] PATH_INFO: `/v3/projects/2dc306f881e6416499720bb4860c43e8/users/70c921d91cc04c24a0466395f2aa29ef/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:42.698341 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b76614-91cc-4534-b7ca-14d843a87118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dc306f881e6416499720bb4860c43e8, user_id=70c921d91cc04c24a0466395f2aa29ef, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:42.701139 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b76614-91cc-4534-b7ca-14d843a87118 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:42.770945 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1459/2914] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:42 2022] PUT /identity/v3/projects/2dc306f881e6416499720bb4860c43e8/users/70c921d91cc04c24a0466395f2aa29ef/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:43:42.775761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab68a7f-1d5d-4004-be7d-9cfea18940cf admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:42.785953 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab68a7f-1d5d-4004-be7d-9cfea18940cf admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:42.843364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88762eaf-c401-4259-b16e-b7a214913832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:42.844111 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88762eaf-c401-4259-b16e-b7a214913832 admin admin] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:42.844385 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88762eaf-c401-4259-b16e-b7a214913832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:42.844649 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88762eaf-c401-4259-b16e-b7a214913832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:42.845290 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88762eaf-c401-4259-b16e-b7a214913832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:42.846893 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88762eaf-c401-4259-b16e-b7a214913832 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:42.856696 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1456/2915] 10.222.0.62 () {58 vars in 1122 bytes} [Mon Sep 5 12:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 12:43:42.860956 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b76614-91cc-4534-b7ca-14d843a87118 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:42.872068 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b76614-91cc-4534-b7ca-14d843a87118 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:42.877155 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd8fa7-3b9c-4fdc-8ac0-1d7103f24cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:42.877824 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd8fa7-3b9c-4fdc-8ac0-1d7103f24cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:42.878104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd8fa7-3b9c-4fdc-8ac0-1d7103f24cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:42.878379 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd8fa7-3b9c-4fdc-8ac0-1d7103f24cd0 admin admin] PATH_INFO: `/v3/projects/2dc306f881e6416499720bb4860c43e8/users/70c921d91cc04c24a0466395f2aa29ef/roles/705907b9bdec47cebf88b868a6a56356` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:42.883273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd8fa7-3b9c-4fdc-8ac0-1d7103f24cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dc306f881e6416499720bb4860c43e8, user_id=70c921d91cc04c24a0466395f2aa29ef, role_id=705907b9bdec47cebf88b868a6a56356)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:42.886287 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd8fa7-3b9c-4fdc-8ac0-1d7103f24cd0 admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:42.898477 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1460/2916] 10.222.0.62 () {60 vars in 1483 bytes} [Mon Sep 5 12:43:42 2022] PUT /identity/v3/projects/2dc306f881e6416499720bb4860c43e8/users/70c921d91cc04c24a0466395f2aa29ef/roles/705907b9bdec47cebf88b868a6a56356 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 12:43:48.282549 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45de976-404b-479e-966b-57273d8a0142 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:48.283129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45de976-404b-479e-966b-57273d8a0142 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:48.283129 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45de976-404b-479e-966b-57273d8a0142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:48.316286 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-c45de976-404b-479e-966b-57273d8a0142 None None] MFA Rules not processed for user `70c921d91cc04c24a0466395f2aa29ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:48.317634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-c45de976-404b-479e-966b-57273d8a0142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:48.370597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:48.401079 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1457/2917] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:48.421445 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd8fa7-3b9c-4fdc-8ac0-1d7103f24cd0 admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:43:48.424399 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1dfd8fa7-3b9c-4fdc-8ac0-1d7103f24cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:48.502955 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd8fa7-3b9c-4fdc-8ac0-1d7103f24cd0 admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:43:48.509081 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78e0af-2a2a-4b55-8bbf-c2fc2fae28b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:43:48.509688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78e0af-2a2a-4b55-8bbf-c2fc2fae28b7 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:48.509970 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78e0af-2a2a-4b55-8bbf-c2fc2fae28b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:48.510273 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78e0af-2a2a-4b55-8bbf-c2fc2fae28b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:48.515356 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:43:48.565806 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78e0af-2a2a-4b55-8bbf-c2fc2fae28b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:43:48.567414 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78e0af-2a2a-4b55-8bbf-c2fc2fae28b7 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:43:48.577886 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1461/2918] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:43:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:43:51.215818 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfc9c8-df78-44c9-9f5e-7c7f5470d681 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:43:51.216310 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfc9c8-df78-44c9-9f5e-7c7f5470d681 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:43:51.216414 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfc9c8-df78-44c9-9f5e-7c7f5470d681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:43:51.235256 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-91cfc9c8-df78-44c9-9f5e-7c7f5470d681 None None] MFA Rules not processed for user `e4315027930746c2b037efa139531914`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:43:51.236507 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-91cfc9c8-df78-44c9-9f5e-7c7f5470d681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:43:51.244217 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1458/2919] 10.222.0.62 () {58 vars in 958 bytes} [Mon Sep 5 12:43:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:43:58.774294 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=24 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:44:01.229769 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=11 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:44:10.254717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78e0af-2a2a-4b55-8bbf-c2fc2fae28b7 service nova] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:44:10.256152 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-1c78e0af-2a2a-4b55-8bbf-c2fc2fae28b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:44:10.312416 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78e0af-2a2a-4b55-8bbf-c2fc2fae28b7 service nova] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:44:10.316498 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2a515-0c76-43b8-b2ee-98f4e94ee2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:44:10.316662 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2a515-0c76-43b8-b2ee-98f4e94ee2eb service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:44:10.316878 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2a515-0c76-43b8-b2ee-98f4e94ee2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:44:10.317151 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2a515-0c76-43b8-b2ee-98f4e94ee2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:44:10.318873 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-33f2a515-0c76-43b8-b2ee-98f4e94ee2eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:44:10.329057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2a515-0c76-43b8-b2ee-98f4e94ee2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:44:10.330500 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2a515-0c76-43b8-b2ee-98f4e94ee2eb service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:44:10.339646 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1462/2920] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:44:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:44:17.004363 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfc9c8-df78-44c9-9f5e-7c7f5470d681 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:44:17.014165 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfc9c8-df78-44c9-9f5e-7c7f5470d681 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:44:17.017753 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3956f-6afc-47a6-93a7-bbc37febe63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:44:17.018346 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3956f-6afc-47a6-93a7-bbc37febe63a service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:44:17.018622 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3956f-6afc-47a6-93a7-bbc37febe63a service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:44:17.018890 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3956f-6afc-47a6-93a7-bbc37febe63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:44:17.020593 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-82f3956f-6afc-47a6-93a7-bbc37febe63a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:44:17.033635 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3956f-6afc-47a6-93a7-bbc37febe63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:44:17.035039 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3956f-6afc-47a6-93a7-bbc37febe63a service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:44:17.046416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1459/2921] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:44:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:44:18.228751 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2a515-0c76-43b8-b2ee-98f4e94ee2eb service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:44:18.237452 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2a515-0c76-43b8-b2ee-98f4e94ee2eb service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:44:18.241096 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daca691-77bc-4bb5-843c-62beaa752ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:44:18.241688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daca691-77bc-4bb5-843c-62beaa752ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:44:18.241966 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daca691-77bc-4bb5-843c-62beaa752ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:44:18.242238 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daca691-77bc-4bb5-843c-62beaa752ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:44:18.244435 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-8daca691-77bc-4bb5-843c-62beaa752ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:44:18.257942 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daca691-77bc-4bb5-843c-62beaa752ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:44:18.259970 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daca691-77bc-4bb5-843c-62beaa752ae7 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:44:18.270747 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1463/2922] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:44:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:44:27.041460 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:44:28.265936 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=32 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:44:30.341011 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3956f-6afc-47a6-93a7-bbc37febe63a service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:44:30.349959 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3956f-6afc-47a6-93a7-bbc37febe63a service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:44:30.354430 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad645b-aaf9-41a6-8ff1-87efabadacec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:44:30.355034 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad645b-aaf9-41a6-8ff1-87efabadacec service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:44:30.357075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad645b-aaf9-41a6-8ff1-87efabadacec service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:44:30.357075 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad645b-aaf9-41a6-8ff1-87efabadacec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:44:30.365373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad645b-aaf9-41a6-8ff1-87efabadacec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:44:30.366830 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad645b-aaf9-41a6-8ff1-87efabadacec service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:44:30.377124 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1460/2923] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:44:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:44:32.281866 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daca691-77bc-4bb5-843c-62beaa752ae7 service neutron] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:44:32.293336 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daca691-77bc-4bb5-843c-62beaa752ae7 service neutron] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:44:32.298989 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de16de-565b-4e60-835c-0eff5b7c542a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:44:32.299620 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de16de-565b-4e60-835c-0eff5b7c542a service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:44:32.299901 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de16de-565b-4e60-835c-0eff5b7c542a service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:44:32.300176 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de16de-565b-4e60-835c-0eff5b7c542a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:44:32.301914 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-15de16de-565b-4e60-835c-0eff5b7c542a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:44:32.314885 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de16de-565b-4e60-835c-0eff5b7c542a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:44:32.316683 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de16de-565b-4e60-835c-0eff5b7c542a service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:44:32.328774 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1464/2924] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:44:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:44:40.372461 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:44:42.323706 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=6 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:45:33.040498 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad645b-aaf9-41a6-8ff1-87efabadacec service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:45:33.050027 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad645b-aaf9-41a6-8ff1-87efabadacec service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:45:33.053791 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45a0a8-66aa-4a4d-a8a4-6b4f81db1415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:45:33.054389 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45a0a8-66aa-4a4d-a8a4-6b4f81db1415 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:45:33.054666 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45a0a8-66aa-4a4d-a8a4-6b4f81db1415 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:45:33.054936 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45a0a8-66aa-4a4d-a8a4-6b4f81db1415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:45:33.056725 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-ef45a0a8-66aa-4a4d-a8a4-6b4f81db1415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:45:33.066700 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45a0a8-66aa-4a4d-a8a4-6b4f81db1415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:45:33.068152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45a0a8-66aa-4a4d-a8a4-6b4f81db1415 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:45:33.076295 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1461/2925] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:45:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:45:43.072602 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:46:44.652520 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc0f38-a819-487d-b492-acf906bafbae None None] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:46:44.653057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc0f38-a819-487d-b492-acf906bafbae None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:46:44.653057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc0f38-a819-487d-b492-acf906bafbae None None] PATH_INFO: `/` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:46:44.654392 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1465/2926] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:46:44.660328 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06195fc3-e280-4571-a548-319597989216 None None] REQUEST_METHOD: `POST` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:46:44.660824 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06195fc3-e280-4571-a548-319597989216 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:46:44.660961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06195fc3-e280-4571-a548-319597989216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:46:44.711645 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.auth.core [None req-06195fc3-e280-4571-a548-319597989216 None None] MFA Rules not processed for user `9a89a2e6aeb4495a84a61bd82b02bfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81649) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:46:44.712846 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-06195fc3-e280-4571-a548-319597989216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:46:44.719524 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1462/2927] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:46:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 12:46:44.733010 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc0f38-a819-487d-b492-acf906bafbae None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:46:44.744067 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc0f38-a819-487d-b492-acf906bafbae None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:46:44.747616 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8a2f9-6485-4acf-be15-dc8dd3bca845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:46:44.748250 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8a2f9-6485-4acf-be15-dc8dd3bca845 service neutron] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:46:44.748531 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8a2f9-6485-4acf-be15-dc8dd3bca845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:46:44.748806 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8a2f9-6485-4acf-be15-dc8dd3bca845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:46:44.756020 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8a2f9-6485-4acf-be15-dc8dd3bca845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:46:44.757444 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8a2f9-6485-4acf-be15-dc8dd3bca845 service neutron] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:46:44.765109 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1466/2928] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:46:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:46:44.848627 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4ee45-ba1e-41f7-943e-177fff7ae942 None None] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:46:44.849428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4ee45-ba1e-41f7-943e-177fff7ae942 None None] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:46:44.849701 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4ee45-ba1e-41f7-943e-177fff7ae942 None None] PATH_INFO: `/` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:46:44.851053 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1463/2929] 10.222.0.62 () {54 vars in 921 bytes} [Mon Sep 5 12:46:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 12:46:44.856510 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07ad20-5037-4b17-bf71-01c5746223ce None None] REQUEST_METHOD: `POST` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:46:44.856795 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07ad20-5037-4b17-bf71-01c5746223ce None None] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:46:44.857073 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07ad20-5037-4b17-bf71-01c5746223ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:46:44.894931 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.auth.core [None req-fb07ad20-5037-4b17-bf71-01c5746223ce None None] MFA Rules not processed for user `7956a7748ec94a618eb466f5536220b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81648) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 12:46:44.896658 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fb07ad20-5037-4b17-bf71-01c5746223ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:46:44.905807 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1467/2930] 10.222.0.62 () {58 vars in 1018 bytes} [Mon Sep 5 12:46:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 12:46:44.916773 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4ee45-ba1e-41f7-943e-177fff7ae942 None None] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:46:44.927615 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4ee45-ba1e-41f7-943e-177fff7ae942 None None] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:46:44.931083 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b1da1-efc6-4e52-976a-c4d28c6992d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a89a2e6aeb4495a84a61bd82b02bfc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:46:44.931714 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b1da1-efc6-4e52-976a-c4d28c6992d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:46:44.931999 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b1da1-efc6-4e52-976a-c4d28c6992d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:46:44.932268 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b1da1-efc6-4e52-976a-c4d28c6992d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:46:44.942838 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b1da1-efc6-4e52-976a-c4d28c6992d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:46:44.944270 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b1da1-efc6-4e52-976a-c4d28c6992d9 service neutron] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:46:44.954597 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1464/2931] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:46:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:46:45.018334 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07ad20-5037-4b17-bf71-01c5746223ce None None] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:46:45.019773 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-fb07ad20-5037-4b17-bf71-01c5746223ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:46:45.088229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07ad20-5037-4b17-bf71-01c5746223ce None None] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:46:45.093318 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49fc4c-6fee-44ef-bf5e-ba0baa891208 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:46:45.093926 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49fc4c-6fee-44ef-bf5e-ba0baa891208 service placement] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:46:45.094206 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49fc4c-6fee-44ef-bf5e-ba0baa891208 service placement] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:46:45.094478 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49fc4c-6fee-44ef-bf5e-ba0baa891208 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:46:45.096224 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ca49fc4c-6fee-44ef-bf5e-ba0baa891208 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:46:45.107454 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49fc4c-6fee-44ef-bf5e-ba0baa891208 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:46:45.109057 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49fc4c-6fee-44ef-bf5e-ba0baa891208 service placement] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:46:45.118849 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1468/2932] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:46:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:46:46.057259 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b1da1-efc6-4e52-976a-c4d28c6992d9 service neutron] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:46:46.069029 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b1da1-efc6-4e52-976a-c4d28c6992d9 service neutron] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:46:46.074761 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc2687-0e6a-4a2b-b9ff-b472d3a00519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:46:46.075364 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc2687-0e6a-4a2b-b9ff-b472d3a00519 service nova] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:46:46.075671 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc2687-0e6a-4a2b-b9ff-b472d3a00519 service nova] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:46:46.075961 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc2687-0e6a-4a2b-b9ff-b472d3a00519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:46:46.077640 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-afdc2687-0e6a-4a2b-b9ff-b472d3a00519 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:46:46.090928 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc2687-0e6a-4a2b-b9ff-b472d3a00519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:46:46.092522 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc2687-0e6a-4a2b-b9ff-b472d3a00519 service nova] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:46:46.103977 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1465/2933] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:46:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:46:55.114688 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=47 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:46:56.098632 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=25 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:48:18.902419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49fc4c-6fee-44ef-bf5e-ba0baa891208 service placement] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:48:18.904111 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ca49fc4c-6fee-44ef-bf5e-ba0baa891208 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:48:18.968644 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49fc4c-6fee-44ef-bf5e-ba0baa891208 service placement] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:48:18.974219 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93b4f2-8ca2-48d7-beb9-40701e13c018 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7956a7748ec94a618eb466f5536220b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:48:18.974817 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93b4f2-8ca2-48d7-beb9-40701e13c018 service nova] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:48:18.975095 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93b4f2-8ca2-48d7-beb9-40701e13c018 service nova] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:48:18.975372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93b4f2-8ca2-48d7-beb9-40701e13c018 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:48:18.977181 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-ed93b4f2-8ca2-48d7-beb9-40701e13c018 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:48:18.989104 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93b4f2-8ca2-48d7-beb9-40701e13c018 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:48:18.990780 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93b4f2-8ca2-48d7-beb9-40701e13c018 service nova] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:48:19.001626 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1469/2934] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:48:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:48:24.729233 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc2687-0e6a-4a2b-b9ff-b472d3a00519 service nova] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:48:24.791148 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc2687-0e6a-4a2b-b9ff-b472d3a00519 service nova] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:48:24.796540 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d78561-e573-485c-9908-fd5ba9ab05a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:48:24.797139 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d78561-e573-485c-9908-fd5ba9ab05a9 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:48:24.797416 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d78561-e573-485c-9908-fd5ba9ab05a9 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:48:24.797695 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d78561-e573-485c-9908-fd5ba9ab05a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:48:24.799428 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-44d78561-e573-485c-9908-fd5ba9ab05a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:48:24.825036 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d78561-e573-485c-9908-fd5ba9ab05a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:48:24.825545 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d78561-e573-485c-9908-fd5ba9ab05a9 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:48:24.836304 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1466/2935] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:48:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:48:28.339245 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93b4f2-8ca2-48d7-beb9-40701e13c018 service nova] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:48:28.421229 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93b4f2-8ca2-48d7-beb9-40701e13c018 service nova] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:48:28.426758 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:48:28.427372 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:48:28.427689 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:48:28.427968 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] PATH_INFO: `/v3/users/e4315027930746c2b037efa139531914` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:48:28.429092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4315027930746c2b037efa139531914)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:48:28.431094 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:48:28.578332 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4315027930746c2b037efa139531914'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:28.578990 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4315027930746c2b037efa139531914'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:28.650574 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4315027930746c2b037efa139531914'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:28.663863 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4315027930746c2b037efa139531914'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:28.664504 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4315027930746c2b037efa139531914'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:28.672895 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:48:28.677092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1470/2936] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:48:28 2022] DELETE /identity/v3/users/e4315027930746c2b037efa139531914 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:48:28.894971 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d78561-e573-485c-9908-fd5ba9ab05a9 service placement] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:48:28.907585 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d78561-e573-485c-9908-fd5ba9ab05a9 service placement] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:48:28.966261 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:48:28.967409 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:48:28.967722 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:48:28.968021 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] PATH_INFO: `/v3/projects/58a867fbd2d444728391be6de51146c9` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:48:28.969114 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a867fbd2d444728391be6de51146c9)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:48:28.971007 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:48:29.073891 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a867fbd2d444728391be6de51146c9', 'request_id': 'req-1fd057a7-a7aa-4154-835b-4c5ac74725b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.079412 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a867fbd2d444728391be6de51146c9', 'request_id': 'req-1fd057a7-a7aa-4154-835b-4c5ac74725b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.165070 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a867fbd2d444728391be6de51146c9', 'request_id': 'req-1fd057a7-a7aa-4154-835b-4c5ac74725b5'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.188882 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] The token cache is being invalidate because project 58a867fbd2d444728391be6de51146c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:48:29.189168 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.189796 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.191373 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1467/2937] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:48:28 2022] DELETE /identity/v3/projects/58a867fbd2d444728391be6de51146c9 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:48:29.195343 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:48:29.196874 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:48:29.209916 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f66e8-bd7c-41c5-9faa-9ec4d8d9afab admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:48:29.265534 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:48:29.266147 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:48:29.266419 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:48:29.266690 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] PATH_INFO: `/v3/users/70c921d91cc04c24a0466395f2aa29ef` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:48:29.268070 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c921d91cc04c24a0466395f2aa29ef)` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:48:29.269983 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:48:29.418508 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c921d91cc04c24a0466395f2aa29ef'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.419157 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c921d91cc04c24a0466395f2aa29ef'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.484657 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c921d91cc04c24a0466395f2aa29ef'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.496671 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c921d91cc04c24a0466395f2aa29ef'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.497319 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.notifications [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c921d91cc04c24a0466395f2aa29ef'} {{(pid=81648) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.509875 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1471/2938] 10.222.0.62 () {58 vars in 1224 bytes} [Mon Sep 5 12:48:29 2022] DELETE /identity/v3/users/70c921d91cc04c24a0466395f2aa29ef => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:48:29.732488 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:48:29.733956 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:48:29.746575 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd057a7-a7aa-4154-835b-4c5ac74725b5 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:48:29.752943 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:48:29.807401 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57929111b78840da8ec77704273d56d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd80f282fe9da41e0b08a14ba58840408', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:48:29.808055 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:48:29.808330 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:48:29.808619 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] PATH_INFO: `/v3/projects/2dc306f881e6416499720bb4860c43e8` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:48:29.809743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dc306f881e6416499720bb4860c43e8)` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:48:29.811751 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:48:29.921692 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dc306f881e6416499720bb4860c43e8', 'request_id': 'req-f9468100-3d57-42b0-bebd-1e2718084950'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:29.927634 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dc306f881e6416499720bb4860c43e8', 'request_id': 'req-f9468100-3d57-42b0-bebd-1e2718084950'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:30.012558 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dc306f881e6416499720bb4860c43e8', 'request_id': 'req-f9468100-3d57-42b0-bebd-1e2718084950'} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:30.023758 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] The token cache is being invalidate because project 2dc306f881e6416499720bb4860c43e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81649) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 12:48:30.024041 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:30.024659 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.notifications [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81649) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 12:48:30.026117 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1468/2939] 10.222.0.62 () {58 vars in 1233 bytes} [Mon Sep 5 12:48:29 2022] DELETE /identity/v3/projects/2dc306f881e6416499720bb4860c43e8 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 12:48:36.520717 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] Authenticating user token {{(pid=81648) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:48:36.522165 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:48:36.591437 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070de76-6923-48d8-b7e0-c2be8bb6fd2c admin admin] Validating token access rules against request {{(pid=81648) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:48:36.608726 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:48:36.651153 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47723d27-22cf-41ba-accb-3fdd834795cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81648) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:48:36.651814 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47723d27-22cf-41ba-accb-3fdd834795cb service placement] REQUEST_METHOD: `GET` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:48:36.652092 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47723d27-22cf-41ba-accb-3fdd834795cb service placement] SCRIPT_NAME: `/identity` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:48:36.652367 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47723d27-22cf-41ba-accb-3fdd834795cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81648) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:48:36.654142 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.fernet_utils [None req-47723d27-22cf-41ba-accb-3fdd834795cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81648) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:48:36.665326 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47723d27-22cf-41ba-accb-3fdd834795cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:48:36.666981 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47723d27-22cf-41ba-accb-3fdd834795cb service placement] RBAC: Authorization granted {{(pid=81648) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:48:36.677572 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: [pid: 81648|app: 0|req: 1472/2940] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:48:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:48:40.020743 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:48:46.696562 dv-cinder-801354-7-iscsi devstack@keystone.service[81648]: DEBUG dbcounter [-] [81648] Writing DB stats keystone:SELECT=22 {{(pid=81648) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 12:53:37.473183 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] Authenticating user token {{(pid=81649) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 12:53:37.474603 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.fernet_utils [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81649) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 12:53:37.489242 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9468100-3d57-42b0-bebd-1e2718084950 admin admin] Validating token access rules against request {{(pid=81649) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 12:53:37.494358 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c136bf-e79e-4286-8ac6-21de60ae2206 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90d65372febb414d82843f9dfa56f738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2c48ceeacd6453ca9af78ca41554230', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81649) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 12:53:37.494951 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c136bf-e79e-4286-8ac6-21de60ae2206 service placement] REQUEST_METHOD: `GET` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 12:53:37.495226 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c136bf-e79e-4286-8ac6-21de60ae2206 service placement] SCRIPT_NAME: `/identity` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 12:53:37.495495 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c136bf-e79e-4286-8ac6-21de60ae2206 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81649) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 12:53:37.504800 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c136bf-e79e-4286-8ac6-21de60ae2206 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 12:53:37.506347 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c136bf-e79e-4286-8ac6-21de60ae2206 service placement] RBAC: Authorization granted {{(pid=81649) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 12:53:37.517152 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: [pid: 81649|app: 0|req: 1469/2941] 10.222.0.62 () {60 vars in 1365 bytes} [Mon Sep 5 12:53:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 12:53:47.512194 dv-cinder-801354-7-iscsi devstack@keystone.service[81649]: DEBUG dbcounter [-] [81649] Writing DB stats keystone:SELECT=6 {{(pid=81649) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}